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ABSTRACT 

This handbook documents assessment strategies that 
are in current use, have potential for duplication, or are in the 
testing and development stages, and that hold promise for more 
accurate assessment of limited English proficient (LEP) individuals 
in vocational programs. The handbook was developed as a result of a 
research and demonstration project funded by the Office of Vocational 
and Adult Education (ED) to assist in the implementation of the Carl 
D. Perkins Vocational and Applied Technology Education Act of 1990. 
The chapters in the handbook define and provide a rationale for 
different issues and strategies of assessment. After an analysis of 
the fundamentals of assessment and the forces that are expanding the 
concept of testing, specific techniques are explained with examples. 
The methods include the following: (1) interviews; (2) activities 
using the Cloze test format: (3) rating scales; (4) portfolios; and 
(5) projer.s. A chapter of fairer usage and clearer application of 
standardized tests is also included, along with a glossary and 
extensive references for each section. (Author/KC) 
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Reld Test Sites 

Our sincere appreciation goes to the dedicated and committed educators we 
worked with during the Field Test stage of our project. Being able to share in 
their experiences provided us with very responsive and helpful insights which 
enabled us to move ahead on a more complete development of this 
Handbook. 

In order to validate the earlier research findings of potentially promising 
assessment practices, it was important to select a small, but representative 
number of sites around the country where we could share our suggestions and 
receive direct and diverse feedback on the appropriateness and usefulness of 
the targeted strategies and techniques. 

At all sites, the basic goal was to establish a dialogue with practitioners and a 
"hands on" exchange of ideas about what is being done and what else can be 
done to have assessment more accurately reflect LEP students' strengths and 
respond to their needs. 

Due to the parameters and limitations of our project, resources were not 
available for comprehensive, in-depth field tests. Therefore, we primarily relied 
upon three to four-day visits to a few representative secondary and post 
secondary schools. 

These included: 



1) Bell Multicultural High School 
Washington, D.C. 

Chief Contact Persons: 
Ms. Ana Vasquez, Director 
Language Arts Curriculum 

Mr. Kevin Farley, Math Teacher & Curriculum Developer 
Tech Prep Program 
Phone: 202-673-3551 
FAX: 202-673-7581 

2) Brevard Community College 
Palm Bay, Florida 

Chief Contact Persons: 

Ms. Kathy Talley, Vocational Instructor 

Ms. Sue Lundgren, VESL Instructor 

Phone:407-951-1060 

FAX: 407-723^*674 
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3) Northside Independent School District 
San Antonio, Texas 

Chief Contact Person: 

Mr. Jadnto Gonzales, Director 

Vocational & Applied Technology 

Phone: 210-647-2232 

Fax: 210-647-2391 

4) Rindge School of Technical Arts 
Cambridge, Massachusetts 
Chief Contact Person: 

Mr. Larry Rosenstock, Executive Director 
Phone: 617-34&-6753 
FAX: 617-349-6770 



In general, these school visits included: 
• Short introductory sessions; 



• Condensed periods of observations of students engaged in 
instruction or assessment activities; and 

• Meetings with vocational teachers, guidance counselors, 
assessment personnel, and administrators. 

In addition to the school visits, we engaged in three additional field test site 
activities which provided us with an opportunity for analysis and evaluation 
of the broad scope of procedures and strategies we targeted. These 
efforts included: 



5) School of Occupational and Educational Studies 
Graduate Seminar 
Colorado State University 
Ft. Collins, Colorado 
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Chief Contact Person: 
Ms. Jean Lehmann, Professor 
Phone: 3aM91-5169 
FAX: 303-491-7204 

Our suggested strategies and techniques were included as discussion 
topics for several weeks in the curriculum of a post-graduate seminar, and 
evaluation forms documented the students' impressions for inclusion in our 
field tests. 



6) El Camino Community College 

Bilingual Vocational Instructors Training Program 

Torrance, California 

Chief Contact Person: 

Ms. Laura Franklin, Director 

Phone: 310-715-7830 

FAX: 310-715-3378 

El Camino Community College sen/es as the coordinating office for a 
federally funded Bilingual Vocational Instructors Training Program (BVIT) 
that involves several community colleges throughout the state of California. 
Our suggested strategies and techniques were shared with several BVIT 
program participants and evaluation forms documented their impressions 
for inclusion in our field tests. 

7) Special Needs/National Association of Vocational Assessment 
in Education 

American Vocational Association National Conference 
Nashville, Tennessee 

Ms. Kristin Fiske, Crosspaths Management Systems, Inc.'s Assistant Project 
Manager, gave a presentation on our suggested strategies and techniques 
to the Special Needs/National Association of Vocational Assessment in 
Education (NAVAE), December 3 - 6, 1993, during the national convention 
of the American Vocational Association. In addition to sharing our findings, 
Ms. Fiske used this occasion to network with other educators who provided 
very helpful feedback and constructive suggestions about our targeted 
assessment modifications. 
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INTRODUCTION 



"The assessments that count - the assessments that most 
strongly influence student learning and academic 
self-concept - are those developed and used by teachers 
in the classroom. Teachers want and need help with these 
assessments. They must have it." 

Richard J, Stiggins,{1988, p. 368) 



ACCURATE and COMPREHENSIVE assessment is an essential element 
in developing the education, skills, and critical learning capabilities of your 
students. APPROPRIATE assessment strategies are probably the most 
important tools you have to ensure your students a gateway to personal 
and professional development. 



They also serve as the benchmark for accountability and measurement of 
progress. Schools and many other institutions in American society rely on 
test performance for a variety of indicators. 



Assessment can be more ACCURATE, COMPREHENSIVE 
and APPROPRIATE within the vocational education setting 
when strategies foster a communal effort to see what best 
works for the student. While this is the goal with all 
students, it is especially critical for the authentic assessment 
of limited English proficient (LEP) students. 
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More ACCURATE, COMPREHENSIVE and APPROPRIATE assessment 
can: 



Ensure LEP learners equal access into programs. 

Imbed assessment in the curriculum as an integral, formative 
activity. 

Provide vocational LEP learners quality academic instruction 
and skills development. 

Prepare LEP learners for the demands of the workplace of 
the 21 St century. 

Enable all individuals to pursue their full personal potential 
and development. 



This Handbook is designed to contribute useful tools to accomplish this 
mission. After familiarizing you with some basic terminology in Chapter 3, 
"Glossary", and providing some assessment fundamentals and general 
helpful hints in Chapter 4, "Understanding the Purpose of Assessment", 
we highlight the following promising authentic assessment strategies and 
techniques: 



Chapter 5 


- Interviews 


Chapter 6 


- CLOZE Formats 


Chapter 7 


- Rating Scales 


Chapter 8 


- Portfolios 


Chapter 9 


- Projects 
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In addition, we include Chapter 10, "Standardized Tests", which shares 
helpful hints on: 



How to know if a standardized test is right for you and your LEP 
students. 



How to make the most of a standardized test. 



In order to provide the information in a quick and easy format, we designed 
all Chapters to answer four basic questions: 




WHAT? 



Defines the strategy 




WHY? 



Puts the strategy in an 
understandable context 
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HOW? 

Describes steps to implement the 
strategy 



WHERE? 

Directs you to individual and 
organizational contacts, and additional 
written information. 



Also, throughout the Handbook, we use the symbol which indicates 
where more detailed information, additional illustrations and examples of a 
particular practice can be found. 



ERLC 



Page 4 



Crosspoths Managomont Systems, Inc. 



14 



Introduction - Chapter 2 



The Handbook 




WHAT? 



This Handbook presents the findings of a research and development 
project, funded by the U.S. Department of Education, which studied 
vocational student assessment practices and their impact on LEP 
individuals. As a result of our research, this resource guide shares 
promising assessment strategies which may enable you to achieve more 
accurate assessment results for your program and student population. The 
strategies and techniques described are intended to assist in the 
implementation of the Carl D. Perkins Vocational and Applied Technology 
Education Act of 1990, (P.L 101-392). 



This Handbook IS NOT: 

• A Quick Fix 

• The Silver Bullet 

• A Definitive Digest Of Vocational Student Assessment 
Procedures 

• A Commercial Assessment Instrument 

• Exciting Leisure Reading !!! 



BUT - - - 
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This Handbook IS a guide you can use to find some answers to: 




WHAT is happening in the field of assessment and 
vocational education? What are practitioners doing to 
try to cope with all the changes that are occurring? 
What can be done to better meet the needs and show 
the strengths of your students, especially when their 
English fluency might be limited? 




WHY is there a need to reexamine how we assess 
students? Why changes have already occurred in the 
field, and why more need to happen? Why do some 
strategies seem to work and others don't? 




HOW can you better understand and implement 
strategies that can keep pace with the changes? How 
can you use assessment to design services to insure full 
participation, making it a diagnostic resource for 
exploring opportunities rather than a label for closing 
doors? How can you establish an on-going assessment 
process for your students, not just a first stab for 
placing people (often a euphemism for "dumping"), a 
grand finale for compiling evaluation statistics? 




WHERE can you find more information about the ideas 
and strategies introduced in this Handbook? Where 
can you go to get a more complete picture, through the 
written word and through other practitioners or pertinent 
organizations? 
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WiHY? 



There is widespread national concern among governmental policy makers, 
leaders in the business and labor communities, educators, and others from 
all sectors, about our country's ability to compete economically in the 
international arena. As the Perkins Act states, educators must now look to 
. . concentrating resources on improving educational programs leading 
to academic and occupational skills competencies needed to work in a 
technologically advanced society". 

In order to compete fully in the expanding global economy, the United 
States must maintain a highly skilled, and retrainable workforce. The jobs 
of the future will require ALL of our workers to have: 

• Strong basic skills 

• Transferable technical skills 

• Higher order thinking skills 



As a result, the Perkins Act requires vocational education programs to make 
improvements with particular emphasis on the needs and requirements of 
special populations which have traditionally been disadvantaged and under 
served by education and training programs. This includes persons who 
require alternative or additional resources, programs, and services in order 
to receive the full range of quality vocational education programs and 
services, such as limited English proficient individuals. 
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It is the intent of federal policy to foster changes in assessment that will 
break past cycles of neglect and disillusionment. 

Assessment: 



Can We Break 
The Proverbial Policy Cycle? 




Return to Neglect Crisis Activity 




with Results 



This Handbook will hopefully provide you with an arsenal of promising 
teacher-student driven authentic evaluation and assessment tools to break 
this cycle. The suggestions are intended to provide you with strategies and 
techniques to help LEP students reach their full personal potential and to 
become productive workforce participants and contributors. 
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HOW? 



The methodology used to produce this Handbook included: 

• In-depth literature review of assessment practices with emphasis on 
those applicable to vocational education programs; 

• informal, wide-ranging discussions and sharing of information with 
practitioners, researchers, and policy makers; 



• On-site visits to select secondary and post-secondary vocational 
education, bilingual, and training programs around the country; 

«ar Preface, Chapter 1, pages 4-6 

• Sharing of impressions and ideas with vocational educators at 
national conferences, and through presentations with small groups 
of educators in other settings; and 



• Assistance of an Advisory Working Group, 
83r Preface, Chapter 1, pages 2-3 

who provided much appreciated input on programmatic, research, 
development and evaluation issues associated with the assessment 
strategies and techniques we reviewed. 
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Conducted October, 1992 - March, 1994, our efforts found two broad 
problems in present assessment practices. They often are: 



(1) Incomplete; they do not touch on all areas which 
should be assessed or they do not allow for full 
expression of the capabilities and needs of the 
student; and 



(2) Inappropriate; they do not measure what is really 
needed or what is relevant to the student. 



Our literature review, discussions with practitioners in the field, and insights 
gained from visits to testing centers and classrooms confirmed that the 
great majority of vocational educators see the value of having more 
complete and appropriate ways to assess LEP students. The need is 
definitely there and growing. 



What seems to be missing is a clearinghouse which documents the 
strategies and techniques that are effectively working or are in the testing 
and Development stages and hold promise to more accurately assess LEP 
individuals. 



This Handbook will hopefully contribute to filling that void and provide the 
catalyst for broad dissemination of potentially useful classroom assessment 
strategies. 
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WHERE? 




Clearinghouses and Technicai Assistance Agencies 



Center on Education and Training for Employment 

Ohio State University 

1900 Kenny Road 

Columbus, OH 43210 

1-800-848-4815 

Formerly the National Center for Research in Vocational Education 

ERIC Clearinghouse on Adult, Career, and Vocational Education 

Ohio State University 

1900 Kenny Road 

Columbus, OH 43210 

1-800-848-4815 

A national information system that identifies, selects, processes, and 
disseminates information on education. Services include microfiche or 
paper copies of materials, reviews and synthesis papers, and computer 
searches. 

ERIC Clearinghouse on Language and Linguistics 
Center for Applied Linguistics 
1118 22nd Street, N.W. 
Washington, D.C. 20037 
(202) 429-9292 

Provides resources to enrich classroom lessons, support research projects, 
develop language programs, and provide infoimation on research and 
practice to educators. Publications and services include mini- 
bibliographies. Questions & Answers, ERIC Digests, ERIC/CLL News 
Bulletins, tailored computer searches, special ERIC/CLL publications, and 
workshop training on ERIC. 
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Intercuitura! Development Research Association 
5835 Callaghan, Suite #350 
San Antonio, TX 78228-1190 
(512) 684-8180 

Non-profit research and public education organization working toward 
eliminating obstacles that minority, economically disadvantaged, and LEP 
students encounter in schools. 

National Center for Research in Vocational Education (NCRVE) 
1995 University Ave., Suite #375 
Berkeley, CA 94704-1058 
(415) 642-4004 

Funded by the U.S. Department of Education Office of Vocational and Adult 
Education to research issues in the field of vocational education. 

National Clearinghouse for Bilingual Education (NCBE) 
1118 22nd Street, N.W. 
Washington, D.C. 20037 
1-800-321 -NCBE 
(202) 467-0867 

Funded by the U.S. Department of Education Office of Bilingual Education 
and Minority Language Affairs, NCBE performs a number of clearinghouse 
activities on issues of relevance to bilingual education. Some of these 
activities include conducting needs assessment activities in addition to 
producing and disseminating monographs on various subjects and other 
products. 

National Tech Prep Clearinghouse of Resources 

National Network for Curriculum Coordination for 

Voc,rrech Education (NNCCVTE), East Central Region 

Sangamon State University, K-80 

Springfield, IL 62794-9243 

Rebecca Douglass-Woodhall, Director 

Phone: (217) 786-6173 

Fax: (217) 786-6036 

Committed to sharing information, curriculum, and resources to assist in 
implementing Tech Prep programs defined in the Carl D. Perkins Vocational 
and Applied Technology Education Act, the Clearinghouse provides 
promotional samples, orientation videotapes, state definitions and 
guidelines, speeches and presentations, curricular planning models, 
articulation agreements, and academic integration curriculum. 
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The Clearinghouse maintains state and local contacts and speaker referrals. 
It is coordinated with the National Center for Research in Vocational 
Education, University of California at Berkeley; Center for Occupational 
Research and Development; Tech Prep Consortium; and U.S. Department 
of Labor. 



The Center 

(formerly) Northwest Educational Cooperative (NEC) 

Bilingual Vocational, ESL, and Adult Education Resource Center 

1855 Mt. Prospect Road 

Des Plaines, IL 60018 

(708) 803-3535 

Publicly funded organization for promoting and disseminating educational 
materials. 



Office for Special Populations - (Formerly Technical Assistance for 

Special Populations Program, TASPP) 

University of Illinois 

338 Education Building 

1310 S. Sixth Street 

Champaign, IL 61820 

(217) 333-0807 

Organization of NCRVE dedicated to educational issues of special 
populations, which include limited English proficient individuals. 




National Network for Curriculum Coordination in 
Vocational and Technical Education 



The following six centers offer a variety of curriculum related services which 
include: collecting and disseminating curriculum related information and 
instructional products; providing library lending services; responding to 
requests for materials and information; providing information for Vocational 
Education Curriculum Materials (VECM) database; conducting VECM 
searches, and providing technical assistance and inservice training. 
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East Central Curriculum Network 

Illinois Vocational Curriculum Center (INVCC) 

K-80 

Sangamon State University 
Springfield, IL 62794 
(217) 786-6375 

Northeast Network for Curriculum Coordination 
New Jersey State Department of Education 
Division of Vocational Education 
Crest Way 

Aberdeen. NJ 07747 
(908) 290-1900 

Western Curriculum Coordination Center 
University of Hawaii at Manoa 
1 776 University Avenue, Wist 21 6 
Honolulu, HI 96844 
(808) 956-7834 

Midwest Curriculum Coordination Center 

Oklahoma Department of Vocational and Technical Education 

1 500 West 7th Avenue 

Stillwater, OK 74074-4363 

(405) 377-2000 

Northwest Curriculum Coordination Center 
Clover Park Technical College 
4500 Steilacoom Boulevard, SW 
Takoma, WA 98499-4098 
(206) 589-5764 

Southeastern Curriculum Coordination Center 

Mississippi State University (MSU) 

P.O. Drawer Research and Curriculum Unit DX 

Missisrippi State, MS 39762 

(601) 325-2510 
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Select OBEMLA Multifunctional Resource Centers 



University of Okiahoma 

Division of Continuing Education and Public Affairs 

535 Constitution Avenue 

Norman, Oklahoma 73037 

(405) 325-1711 

MRC Director: Hai Iran 

Special Information Area: Counseling for LEP Students 

Interface Network, !nc. 
4800 Southwest Griffith Drive 
Suite 202 

Beaverton, OR 97005 
(503) 644-5741 

MRC Director: Esther Puentes 

Special Information Area: Career Education Programs for LEP 
Students. 

ARC Associates, Inc. 
310 Eighth Street 
Suite 31 1 

Oakland, CA 94607 
(415) 834-9455 

MRC Director: Sau Lim Tsang 

Special Information Area: Bilingual Education for New 
Immigrant/Refugee LEP Students. 

Interface Network, Inc. 
3650 Lake Otis Parkway 
Suite 102 

Anchorage, Alaska, 99501 
(907) 563-7787 

MRC Director: Richard Littlebear 

Special Information Area: Bilingual Vocational Education 
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Relevant Studies on Limited English Proficient 
Students in Vocational Education 



American Vocational Association, The Carl D, Perkins Vocational and 
Applied Technology Act of 1990:The Final Regulations . (1992). Alexandria, 
Virginia: Author. 

Bilingual Education Multifunctional Resource Center, Sen/ice Area 4, Serving 
Limited English Proficient Students in Vocational Education/Career 
Education Programs . (1993). Norman, Oklahoma: Author. 

Callejas, Juan J. The Career Guidance and Counseling of In-School 
Hispanics: Some Practical and Theoretical Considerations. (1985). ERIC 
Document ED#265 269. 

Center for Law and Education, Components of a Good State Plan for 
Vocational Education Under the 1990 Perkins Act . (1991). Washington, 
D.C.: Author. 

Ciccone, J. and Friedenberg, J., Vocational Special Needs Learners: Five 
Years of Research and Development . (1988). Columbus, Ohio: Center on 
Education and Training for Employment. 

Chinatown Resources Development Center, Promising Programs and 
Practices: Vocational Education for Limited-English Proficient Students. 

(1985) . San Francisco, California: Author . 

Edgewood Independent School District. Intercultural Development Research 
Association, Bilingual Vocational Education Needs Assessment Project. 

(1986) . Columbus, Ohio: ERIC Clearinghouse on Adult, Career, and 
Vocational Education. 

ERIC Clearinghouse on Languages and Linguistics. Vocational English as 
a Second Language, (1988). (Computer search Reprint No. 918). 
Washington, D.C.: Center for Applied Linguistics. 

ERIC Clearinghouse on Adult Career, and Vocational Education. Bilingual 
Vocational Education for Immigrants. (ERIC Digest No. 49). Columbus, 
Ohio: Center on Education and Training for Employment. 
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Fieischman, H. L, Models of Vocational Training for Limited English 
Proficient Adults . (1988). Washington, D.C.: U.S. Department of Education. 

Friedenberg, Joan, The Condition of Vocational Education for Limited 
English Proficient Persons in Selected Areas of the United States . (1987). 
Columbus, Ohio: National Center for Research and Vocational Education. 

Grognet, Allene, "Work Skills and the Limited English Proficient Student", 
(1990). Washington, D.C.: Center for Applied Linguistics. 

Harris, W., Improving Vocational Guidance and Counseling Through Career 
Awareness. Orientation and Exploration. (1987). ERIC ED 288 102. 

Kallembach, S., Burac, Z., Coyle-Williams, C, Benesch, J., Bullock, C, and 
llliff, L, Carl D. Perkins Vocational and Applied Technology Education Act 
of 1990: Selected Resources for Implementation . (1992). Berkeley, CA: 
National Center for Research in Vocational Education. 

Kallembach, S., Students with Limited English Proficiency: Selected 
Resources for Vocational Preparation . (1990). Berkeley, California: National 
Center for Research in Vocational Education. 

Kallembach, S., Burac, Z., Coyle-Williams, C, Benesch, J., Bullock, C, and 
llliff, L, Students with Limited English Proficiency: Selected Resources for 
Vocational Preparation. Vol. 2 . (1992). Berkeley, California: National Center 
for Research in Vocational Education. 

Lara, J. and Hoffman, E., School Success for Limited English Proficient 
Students: The Challenge and State Response. (1990). Washington, D.C.: 
Council of Chief State School Officers. 

Lopez-Valadez, J., Immigrant Workers and the American Workplace: The 
Role of Vo' ^?iti -;nal Education . (1985). Columbus, Ohio: Center on 
Education and Training for Employment. 

Lopez-Valadez, J., Training Limited English Proficient Students for the 
Workplace: Trends in Vocational Education. (1989). (New Focus, 
Occasional Paper Series No. 11). Washington, D.C.: The National 
Clearinghouse for Bilingual Education. 



Croerp'^J^o Marwgemorrt Sy sterns, Inc. 



Page 17 



Introduction - Chapter 2 



Martella, J., A Concern about Educating Limited English Proficient Students: 
A CCSSO Survey of State Education Agency Activities, (1989). 
(CONCERNS 26.) Washington, D.C.: Council of Chief State School Officers. 

Michigan State Board of Education, Michigan's Model for Delivering 
Vocational Education to Secondary Limited English Profic e nt and Minority 
Language Students. Volume I: The Model. (1985). Lansing: Author. 

North Carolina State Department of Community Colleges, North Carolina 
Vocational Education Performance Report. Program Year 1 987-1 988. (1 989) . 
Raleigh, North Carolina: Author. 

Phelps, LA., Wermuth, T.R., and Crain, R., Vocational Education for Special 
Populations: Recommendations for Improving State Policy . (1991). 
Berkeley, CA: National Center for Research in Vocational Education. 

Piatt, E., Collaboration for Instruction of LEP Students in Vocational 
Education, (1992). Berkeley, California: National Center for Research in 
Vocational Education. 

Pautler, Jr., A J. (Ed.), Vocational Education in the 1990s: Major Issues , 
(1990). Ann Arbor, Ml: Prakken Publications. 

Quality Education for Minorities Project, Education That Works: An Action 
Plan for the Education for Minorities . (1990). Cambridge, MA: Author. 

Wirt, John, "A New Law on Vocational Education: Will Reform Follow?" Phi 
Delta Kappen , February, 1991. Volume 72. 
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GLOSSARY OF TERMS 



ACCESS ASSESSMENT 



Evaluation process that is used to 
determine entry into a class or pro- 
gram, as well as the level of entry. 



ACCULTURATION 



A process of intercultural borrowing 
between diverse peoples resulting in 
new and blended patterns; acquiring 
the culture of the society one lives in. 



APTFTUDE 



Capacity, inclination for learning. 



ASSESSMENT 



The process of "sitting beside", putting 
a value on an activity and learning 
experience. 



ASSIMILATING 



Absorbing the cultural tradition of a 
population or group. 



AUTHENTIC ASSESSMENT 



Assessment that relies on real-life, 
active applications of knowledge to get 
more accurate and relevant results; 
closely integrated with instruction and 
element of self-assessment. 



BVfT 



Bilingual Vocational Instructors Training 
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BVT 



Bilingual Vocational Training 



CHECKUST 



A guide to follow for quickly seeing 
whether something has been done or 
not. 



CLASSIC CLOZE 



The most common form of the CLOZE 
test; presents text with every few words 
randomly deleted. 



CLOZE 



An assessment technique which alters 
a written text by deleting words and 
calls on the reader^s comprehension 
processing to put the text back into its 
correct form. 



CLOZE EUDE 



Also known as the "CLOZE-edit" or 
"intrusive word technique"; a form of the 
CLOZE test that alters a given text by 
inserting extraneous words. 



COGNITION 



The act or process of knowing, 
including both awareness and 
judgement. 



COMPETENCY-BASED 
ASSESSMENT 



Assessment that gives results in terms 
of the degree of accomplishment of 
stated objectives. 
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Glossary - Chapter 3 
An uninterrupted ordered sequence. 



CRPTERION 



A standard on which a judgment or 
decision can be based. 



CRITERION-BASED 
ASSESSMENT 



Assessment that measures a person's 
performance in reference to 
established, carefully defined 
standards. 



C-TAP 



Career-Technical Assessment Project, 
an assessment project of the California 
Department of Education. 



CTEST 



A format of the CLOZE test which 
deletes the last part of every other 
word. 



CURRICULUM-BASED 
ASSESSMENT 



An on-going evaluation of a student's 
progress towards the objectives of a 
specific course curriculum. 



DEMOGRAPHICS 



The statistical description of 
populations. For this Handbook, 
demographics specifically refers to the 
ethnic, racial, and cultural 
characteristics of populations. 
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DIRECT ASSESSMENT 



Assessment that is designed to answer 
specific, practical, and iminediate 
questions. The student carries out 
steps to demonstrate a skill, and the 
teacher observes and rates how the 
student is performing. 



ESL 



English as a Second Language, Refers 
to a program that teaches English to 
persons whose native language is not 
English. 



EXCELLENCE 



Surpassing in accomplishment or 
achievement. In this Handbook, used 
to define "an institution where all people 
help others become something more 
than they ever hoped to be." 



EXrr ASSESSMENT 



Evaluation process used to determine if 
a student has successfully completed 
the requirements of a particular 
program. 



GESTALT 



School of psychology which studies 
perception and behavior; refers to the 
theory that the human brain naturally 
works to define a unified whole, a 
patterned configuration that has been 
placed together. 
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GRAPHIC RATING 



A linear representation of choices that 
suggests a continuum with equal 
intervals. 



INDIRECT ASSESSMENT 



Assessment that is done through the 
typical pencil and paper, multiple 
choice type of tests. It gives results 
from which you can infer whether or not 
a student would actually be able to 
perform if faced with certain tasks. 



INTERVIEWS 



A form of assessment that is a direct 
manner for gathering important 
information; usually refers to a meeting 
at which information is obtained from 
another person through a two-way 
communication process. 



LEP 



Limited English proficient individual; 
a person whose native language is not 
English, and who has yet to master 
English fluency. 



METACOGNmON 



The process of self-monitoring, being 
able to objectively see and judge one's 
self in relation to learning. 
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NORM-REFERENCED 
TESTS 



Tests that give scores that can be 
compared to how a studied group 
normally performs on the tests. They 
show if the test taker performs above 
the average or below the average of 
what other people have usually done 
on the same test. 



NUMERICAL RATINGS 



Measuring how well a task or behavior 
is being done on a continuum of 
possible levels, using numbers to 
signify the various ranges of 
possibilities. 



OBEMLA 



Office of Bilingual Education and 
Minority Language Affairs, a division of 
the U.S. Department of Education. 



ON-GOING ASSESSMENT 



Evaluation that is used throughout 
various stages of a program to 
determine the progressive levels of a 
student's achievement and knowledge 
being gained. 



PERFORMANCE 
ASSESSMENT 



Evaluates the student's ability to 
apply skills that closely approximate 
those used in real life. Students 
demonstrate competencies by 
performing tasks through procedures 
that are practical applications of 
knowledge. 
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PERKINS ACT 



The Carl D. Perkins Vocational and 
Applied Technology Education Act of 
1990 (Public Law 101-392), the federal 
law which authorizes vocational 
education programs nationwide. 



PORTFOUOS 



An assessment/teaching technique that is 
an assembly of products which demon- 
strate a learned process that has led to 
acquired skills over a period of time. 



PROCESSFOUO 



What some people call a portfolio in 
order to emphasize the importance of 
showing growth and development 
through a collection of work. 



PROJECTS 



An approach for assessing a student's 
on-going progress and acquired skills; 
usually refers to students performing 
specific tasks which lead to a finished 
product. 



RATING SCALE 



Indicates how well a task or behavior is 
being done on a continuum of possible 
levels. It marks at what stage, or to 
what degree, something is being done, 
giving depth to the information that can 
be gathered through this assessment 
technique. 
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RATIONAL DELETION 



A form of the CLOZE test which deletes 
from a written text words central to the 
concept being targeted for assessment. 



RELIABILITY 



The extent to which a test statistically 
yields the same results on repeated trials. 



SCANS 



SCANS, the Secretary's Commission 
on Achieving Necessary Skills, is the 
acronym for U.S. Department of Labor 
funded studies of the late 1980s. 
The appointed Commission members, 
representing educators, business- 
people, labor leaders and others, 
researched the education and skills of 
workers and called for a better prepared 
workforce for the 21st century. 



SHADOWING 



An experiential job assessment strategy 
that allows a student to observe a 
worker doing his/her job duties in the 
workplace; can also be used to 
evaluate classroom courses through 
observing students and teachers. 



SITUATIONAL ASSESSMENT 



The ultimate in "hands-on" assessment, 
where a student is in an actual work or 
training setting for the express purpose 
of assessing an individual's interests, 
abilities and needs. 
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STANDARDIZED TESTS 



Tests which are based on a systematic 
sampling of behavior, have data on 
reliability and validity, and are ad- 
ministered and scored according to 
specific instructions. 



VALIDITY 



Refers to whether or not a test 
actually measures what it intends 
to measure. 



VESL 



Vocational English as a Second 
Language. 



VOCATIONAL ASSESSMENT 



A more traditional, pencil and paper 
approach, emphasizing verbal cues 
and content, to gather information to 
help a student make appropriate 
vocational choices and be able to 
realize them. 



VOCATIONAL EVALUATION 



A hands-on, experiential approach, 
using work-related activities to gather 
information to help a student make 
appropriate vocational choices and be 
able to realize them. 
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ASSESSMENT 



WHAT? 



"Clear definitions are necessary in discussing and describing 
vocational assessment...Unfortunately, however, no consensus 
is apparent in the literature." 

- Peterson, 1988, p.1 



To see what is really meant by "assessment", we should strip away the 
added trappings that have been hung on the word. Let's get back to the 
basics and take it from what originated the whole concept in the first place. 
If we can understand where the idea actually came from and what it was 
meant to be, then maybe we can better understand where it can lead us. 

TO ASSESS.... 

• comes from the Latin word meaning 'to sit beside", and 
referred to proximity in a judge*s office, in other words, you 
are physically close to someone who is making a judgement 
about others. 



• is "to appraise", an evolution from the old French word for "to 
prize" and the Latin word for "to learn", in other words, you 
are putting a high, positive value on an activity th'tat was 
conceived as a learning experience. 
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Seeing the origins of the word 'to assess" shows clearly that the process 
was intended to be a positive learning experience, getting something of 
value on a personal, first hand basis. It is also clear how much of that 
original meaning has been lost by our modernized, fast paced, and mass 
produced assessment techniques. 



Part of the reason there are so many definitions of assessment is that there 
are so many different types. Some overlap, with maybe just one small 
variant feature creating a whole new name for a rehashed idea. Some 
represent opposite ends of the spectrum, reflecting polar points of view 
about the purpose of assessment. 

In our explanations that follow, we define assessment practices that 
represent the whole continuum. 



Some people see "vocational evaluation" as an extension rather than an 
alternative to 'Vocational assessment'. The struggle to make a distinction 
between the two approaches is a leading example of how the field needs 
a clearer understanding of what assessment should cover. 

Vocational Assessment is a gathering of information to help a student make 
appropriate vocational choices and be able to realize them. The way that 
it is usually conducted is also increasingly being referred to as the 
'traditional", ol i way of measuring a person's performance and potential in 
vocational programs. 



The regular routine usually consists of pencil and paper tests covering 
"aptitudes", achievement, interests, etc. The emphasis is definitely on verbal 
cues and content, leading to one conclusion or answer; in other words, 
calling on functions performed by the left brain hemisphere. 
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Vocational Evaluation is a more "hands-on", experiential approach to 
assessment. Using work-related activities as the main basis for the 
assessment process, it calls more on the right brain hemisphere for 
responding. 



Indirect Assessment ... Direct Assessment 



These are two diametrically opposite approaches to assessment, but not 
necessarily mutually exclusive ones, in fact, ideally the assessment process 
should combine tools from both, with careful and cautious interpretation 
done in each case. 

As in the preceding examples of Vocational Assessment and Vocational 
Evaluation, we lead with the side of the spectrum that is the most traditional. 

Indirect Assessment is done through the typical pencil and paper, multiple 
choice type of tests. Students select answers from given possibilities. It 
gives results from which you can infer whether or not a student would 
possibly be able to perform certain tasks. 



In other words, indirect assessment methods yield indications about 
potential performance. 



Direct Assessment is designed to answer specific, practical, and immediate 
questions. This process does that through directly assessing a skill or task 
while the student actually performs or demonstrates it. 



In other words, direct assessment has the student and the teacher 
actively doing things. The student »s carrying out steps to demonstrate 
a skill, and the teacher is observing and rating how the student is 
performing. 
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The example given in Hoachlander's handbook (1992) clearly illustrates the 
difference in these two approaches to assessment. 

Let's say that you want to assess an aspect of a student's auto mechanics 
skills: 



• Direct Assessment: Have the student actually change the oil in a 
car. You would be directly determining whether or not the student 
can perform the specified task. 



• Indirect Assessment: Have the student answer multiple choice 
questions about the procedures involved in changing the oil in a car. 
The score the student receives on the written test would indicate 
whether or not the student could possibly perform the task. 



Many of the terms that are becoming so popular when referring to 
assessment stem from refining what is involved in Direct Assessment. The 
drive towards a more real and relevant direct assessment process has 
created a bandwagon movement with a possibility of models. Some of the 
more prevalent offsprings of direct assessment are: 



Cuniculum-based assessment is an on-going evaluation of a student's 
progress towards the objectives of a specific course curriculum. It uses the 
class material to be learned as the basis for determining the degree to 
which it has been learned. It relates assessment information directly to 
instructional tools, giving both student and teacher continuous feedback on 
what has and has not been mastered in the classroom. 
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Performance-Based Assessment 



Performance-Based Assessment evaluates the student's ability to apply 
skills that closely approximate those used in real life. Students demonstrate 
competencies by performing tasks through two possible procedures: 

1) The tasks are broken down into many small steps and 
students are graded on ability to perform each small step, or 

2) The tasks are complex ones that can be approached in a 
variety of ways and are typically scored holistically. 

Examj. 'es of Performance-based Assessment: 

• NOCTI (National Occupational Competency Testing Institute) 
Cabinet-Making Test 

• experiments 

• essays 



Authentic Assessment 



Authentic Assessment takes performance assessment onto another level. 
It involves performing tasks that cover a wide range of activities, extend 
over time, and include self-assessment. The assessment tasks are closely 
integrated with instruction, and, in the case of vocational education, an 
integration of academic and vocational problem-solving skills. As one 
proponent said: 



"In this approach, assessment drives instiruction, and instruction 
drives assessment, much the way the front and rear axles impel 
one another in a vehicle wrth four-wheel drive." 

- Maeroff, 1991, p. 275 
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Examples of Authentic Assessment: 

• writing reports 

• designing and constructing a mock-up 

• assembling a portfolio 



Situational Assessment 



Situational Assessment is the ultimate in "hands-on" assessment, situating 
the student in an actual work and/or training setting for the express 
purpose of assessing an individual's interests, abilities, and needs. For LEP 
students, this offers the added advantage of employment exposure and 
possibilities, of taking another step in the acculturation process needed to 
fully function in an American workplace. 



WHAT Areas Should Be Explored in The Assessment 
Process? 



What you assess is determined by what you want to know about a student. 
From the vocational education perspective, that means knowledge about 
how students can grow through an educational training program that 

• responds to a student's needs; 

• taps a student's strengths; and 

• enables a student to function at full capacity in a flexible 
workforce. 

In assessing LEP students, there can be so many unknowns or things 
unfamiliar to you and the student, that the process involves more than what 
you want to know. 

It's also what you need to know. 




Page 6 



Crosspaths Management Systems, Inc. 



Assessment - Chapter 4 



Studies of the changing workforce have shown what is needed for full 
employability possibilities. 

By Pages 8-1 1 of this Chapter give a synopsis of these studies. 

When the demands of the modern workforce are examined and connected 
to the particular cultural/linguistic dimensions of LEP individuals, the 
foundation components of a vocational assessment process emerge as: 

• Basic Academic Skills (Reading, Writing and Mathematics in 
both English and native language). 



• Oral Communication Skills (Listening and Speaking in both 
English and native language). 



• Occupationai Skills, Aptitudes, and Knowledge (Includes not 
only prior job knowledge and skills, but also career 
awareness and job seeking skills in this country). 



Fiexibility/Adaptability (Initiative, motivation, and in the case of 
immigrant LEP persons, the level of acculturation into 
American workplace and expectations). 



• Self-Management and Setf-Awareness (Metacognition, 
including interests, learning styles, and organizational skills, 
such as setting goals, fulfilling deadlines, etc.). 



Interpersonal Skills (Teamwork, appropriate behavior in 
different professional/social situations, which can be very 
culture-based activities difficult for immigrant LEP students). 

Problem SoMriy Skills 

Special Needs (Financial, situational, linguistic, etc.). 



Crosspaths Managomont Systems, Inc. Page 7 

44 



Assessment - Chapter 4 




WHY? 



** Age and experience teach us that life doesn't present 
itself in a series of five multiple choices." 

- Alan Wurtzel, CEO Circuit City Stores 



Interest in innovation and reform tends to rise with public anxiety about the 
nation's schools. In vocational education, the interest and anxiety come 
from a double whammy source of concern. Vocational education is part of 
a whole educational system that has "a nation at risk" AND it is precisely 
the part of the system that is supposed to directly address the crisis of an 
inadequately prepared workforce. 




WHY We Need to Change the "Old" Assessment Mind 
Set 



I. Changing Workforce Demands 



Schools, businesses, government, and research organizations are calling 
for a better prepared workforce. During the early development work on this 
Handbook, we reviewed some of the prominent studies that examined the 
"essential elements" needed to maintain a competitive edge. We then 
synthesized the points of convergence into what seemed to represent a 
common core of skills. Selected sources felt to be the most relevant and 
representative were: 



Page 8 



Crosspoths Managomont Systems, Inc. 



Assessment - Chapter 4 



• American Society for Training and Development: America and 
the New Economy , by Anthony Carnevale; 

• SCANS: Learning a Living: A Blueprint for H igh Performance: 
and 

• Experiences of school systems in the state of Michigan and in 
FLWorth, Texas. 

The following graphic shows the results of our analysis, with the shaded points 
of integration forming a common core of skills. They are the areas of 
convergence seen as the common denominators for building a better 
workforce. They also encompass the same components of the vocational 
assessment process we discussed earlier in this chapter. 

»ar Page 7 ol this Chapter 

Common Core of Skills 
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If an educational program is to enable students to succeed in the changing 
world outside the classroom, instruction must also change the dynamics 
and particulars of what is taught and experienced. And, assessment is an 
integral aspect of instruction. 



When it comes to instructing LEP students in highly demanded 
communication sl<ills, it is useful for teachers to see what aspects of the 
complex process are required in the "average" job in America, and to what 
degree. 



Communication on the Job 



Breakdown of Essential Skills 



Listening 
55% 



Speaking Reading 
23% 13% 




Writing 
8% 



Source; Carnevale, ASTD, 1991 
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Notice in the preceding pie chart that speaking is almost one-fourth of a 
working day's communication needs. But also notice in the following chart 
how the spoken message is transmitted through a complex and culturally 
determined process. Learning just the proper words isn't the essential key 
to oral understanding. 



Speaking Skills Needed for Work 

How Spoken Information is Transmitted 



Vocal Skills i^^^^i P'tch & loudness) 



38% 




Language 
7% 



Body Language 
55% 



Source: A. Carnavale, ASTD, 1991 
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II. Changing Demographics 

The results of the 1 990 Census show the changing population realities the 
country is experiencing on the job and at home: 

• During the 1990s, over 40% of new entrants into the work force will 
be minorities. 



• Almost two-thirds of the new entrants into the work force from now 
until the year 2000 will be women. 



• It is estimated that by the year 2000, there will be approximately 1 7.4 
million LEP individuals in the U.S. This will be a 32% increase since 
1980 versus a 17% increase in the general population. 



• The just cited figure of 17.4 million is a very rough estimate, since 
states differ in their definitions of what constitutes "limited English 
proficiency." A clearer number comes from answers to the question 
of how many people live in homes where another language is 
spoken. 

The 1990 Census found that 32 million people in the U.S. speak a 
language other than English at home. This is an increase of 38% 
since 1980. 



The number of LEP people in the U.S. will increase by 1 million a 
year. This increase will not come mainly from immigrants, but rather 
from people born in the U.S. within LEP families. 



• An even more recent study showed that in 1 991 , the largest and 
fastest group of legal immigrants came from countries formerly part 
of the USSR (Chisman et.al., 1993). 
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Native Spanish speakers from various countries make up 60% of the LEP 
population, Dut there are a wide range of other languages represented as 
well. Here you can see a graph of the most prevalent languages other than 
English spoken in U.S. households. The numbers used are from the 1990 
Census survey. 
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With the daily realities of "global communications" and a "global economy" 
making us aware of how the world is becoming increasingly interrelated, it 
is important to put the concepts of language "minority" and '"foreigners" into 
perspective. The world is rapidly changing and Americans must view them- 
selves in the appropriate and more realistic context of the "global community." 



Changing Demographics 

Projected Population for the Year 2025 



Aaia 
58% 




Source: The World Resources Institute, 1990. 
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lis. Cogn'ftive Research Changing Our Understanding of 
How We Learn 



Looking at the U.S. as part of a global, growing community gives another 
reason why the old "assessment mind-set" must be changed. Cognitive 
science researchers call it "cognition"; educators with common sense who 
have been in the classroom oDserving how students learn differently could 
call it "wake up and smell the coffee". 

Anyone interested in the different ways that a person learns is always 
asking questions such as: 

• How does the mind take in information? 



• What does the mind do with the information once it is there? 



• How does the memory store it? 

• What makes us able to apply and amplify that which we have 
stored? 



• And why does it differ so m.uch from student to student? 



The person asking those questions knows that the answers don't come in 
neat packets of tests, made in America. 

Yet, a big part of the traditional assessment mind-set is a measurement 
mentality, a belief that tests can quantify the concepts in learning into exact, 
static numbers. 
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This measurement mentality transforms 
students into passive measuring cups. It 
makes us look at learners as pre-constructed 
receptacles into which we can merely pour 
knowledge. Assessment becomes quick looks 
to see if we have put enougn contents to 
reach the prescribed amount, and when we 
reach a certain level, decide that we have a 
completed product. 



That simplistic measurement approach produces even more questionable 
results within the reality of the world being a global community. 



For example, the United States is one of the very few places on the entire 
planet that still uses the awkward and arbitrarily determined non-metric 
system. If a measuring cup is made abroad for international use, it uses 
grams, not ounces. 

Trying to use our parochial cup in our little corner as we have always done 
creates confusion and misleading results. 



Our students are increasingly international; companies and life styles have 
expanded around the globe. We must meet these changes directly. The 
way we assess our students should incorporate change and we should 
follow 'The Big Picture". 
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WHY The Teacher Is So Important in The New 
Assessment Approaches and Educational Process 



The 1990 Perkins Act changed the way vocational education programs get 
their funding and make decisions. Many see the Act as another example 
of how policy making is getting pushed more towards the local level, closer 
to the people actually involved. 



There are predictions and increasing evidence that the real changes in 
education will come 



NOT from the Capitol, 

NOT from the Courthouse, 

NOT from the City Hall, 

BUT FROM THE CLASSROOM. 

This "trend towards the teacher"* was verified in a study of the many new 
assessment instruments emerging in response to the educational, 
economic, technological and social changes taking place. Of the hundreds 
of different ways being developed to try to more accurately assess the 
learning process and potential of students, three comimon features were 
found to distinguish the most promising ones, (Chittenden, 1991). 



The following graphic lists the three common features the survey found to 
be distinctive of the majority of the good assessment techniques being 
newly developed. Notice that two out of the three common denominators 
lead the process: 

DIRECTLY To The TEACHER 
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1 991 Survey of Varying Attempts 
at Reorienting Assessment Found 
3 Common Features 

1. Capitalize on actual work of the 
dassroom 



Z Enhance teacher and student 
involvement in evaluation 



3. Meet some of the accountability 
concerns of the district 



The results of new assessment approaches pointing the responsibility 
towards the teacher probably won't surprise teachers who have stopped 
to think how they spend their time in the classroom. Grading papers, 
making tests, orally questioning students, physically observing and 
analyzing how students respond are pieces of the assessment process that 
are standard parts of a teacher's day. 




TEACHER 

tho 'Front Line Manager* 



We can put those activities into statistical percentages based on a nation- 
wide study that was done by researchers actually going into classrooms 
(Stiggins, 1988). 




A Teacher's Time In The Classroom 



0 A teacher makes an interactive, evaluative decision every 
2 minutes. 

© An average of 30% of a teacher's time is spent directly in 
assessment activities. 
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You just saw a graphic showing how much of a teacher's time in the 
classroom is spent on assessing students in some form or another. The 
same study that proved that teachers spend a sizable percentage of their 
time with assessment activities also asked the teachers how they learned 
the assessment strategies they use. The key sources of assessment 
information cited by teachers were: 

• other colleagues. 

• their own experiences as students. 

They almost never cited teacher training. 



The study then looked at teacher training programs and confirmed a rather 
unsettling reality with regards to teaching teachers how to assess: 



• Most teacher training programs don't require a course in 
educational measurement for graduation. 



• Many teacher training programs don't even offer a course on 
assessment. 



If a program does offer a course on assessment, it is usually 
just one of statistical analysis to understand pencil and paper, 
standardized tests. 



In other words, when teachers are being taught how to teach, daily 
assessment techniques are largely ignored. 
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Teachers and LEP Students 

When it comes to teachers having the guidance and preparation needed 
to know how to evaluate the differences in dealing with LEP students, the 
situation gets even more frustrating and overwhelming for many. In global 
numbers, this graph shows the statistics on states with the greatest number 
of LEP populations. 



State-by-State 
Estimated LEP Populations 
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(Source: U.S. Department of Education, 1992.) 
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In a closer-to-home study, the National Association for Bilingual Education 
yielded the following breakdown: 



• 1 out of 7 children going to public schools in the U.S. 
are from homes where English isn't spoken. 



• 1 out of 6 teachers have LEP students enrolled in class. 

BUT 

• ONLY 1 OUT of 16 teachers have even had a minimum 
3 hour course in multicultural or bilingual issues. 



Obviously, it can be a very frustrating situation for all involved, and it 
certainly makes you appreciate the comment of another professional who 
had to go to bat many times against the odds: 



••Experience is a hard teacher because she gives the test first and 
the lesson aftenwards." 

- Vernon Law, former pitcher for Pittsburgh Pirates baseball team 



So for those of you who might feel frustrated and confused at times, take 
heart. You're not alone ! 
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To further complicate the matter, the changes happening in assessment 
have ramifications that stem from far-reaching questions about broader, 
underlying principles. Many of the new instruments being developed are 
actually new ways at looking at what and who drives the whole testing 
process. 



Changes in assessment become like "the tail that wags the dog"; seemingly 
small shakes of one part actually rock the entire system. When one questions 
assessment, much more is being questioned, tt also throws into flux: 



• curriculum; 



• teacher training; 



• programs* reasons for being; and 

• entire philosophies in education. 



Highlighting just three of the underlying issues involved illustrates the 
dilemmas and depth of confusion created by changing the traditional 
"assessment mind-set." Doing so brings the educator face to face with 
dealing with a need to: 



1) Clarify the goals of a program 



2) Clarify the use of assessment 



3) Refocus the control of assessment 
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1. Clarifying the Goals of a Program 

What a program aims to achieve determines what factors are assessed and 
established as the accepted measures of "success" or 'failure" of that 
program and the people in it. 

A sun/ey done of vocational education programs found that there was a 
general lack of consensus about what common purposes should be 
addressed. The author of the survey summed up the confusions he found 
in the field: 



"Clearly, the most critical dilemma for vocational education, particularly 
at the secondary level, is achieving consensus... regarding the purpose(s) 
of vocational education and the associated learning outcomes." 

- Phelps, 1991, p. 38 



II. Clarifying the Use of Assessment 



After the 1990 amendments to the Perkins Act became law. NCRVE 
published a "guidebook" for institutions to follow in compliance with the new 
accountability mandates. The book gives assessment suggestions that 
reflect the required changes in the interpretation and control of evaluation 
measures. In discussing the impact of using assessment results in a 
different light, the book's author recognized one of the bottom line points 
in the whole process: 
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" I think at both the secondary and post-secondary level, there has 
never been a mind-set that [tests'] data are something you can use to 
improve programs. Data has always been something you have to 
report to the State to get reimbursed, or to the Feds...it does reflect a 
mind-set 

I think most educators are just not accustomed to thinking about how 
to use [assessment] data to improve what they are doing-** 

- Hoachlander, 1991, p. 2 



111. Change in Control of Assessment Process 



The changing workplace is calling for self-management skills as one of the 
basic "survival" tools needed to succeed. Educators have given the word 
" metacognition" to the process of becoming self-aware and organized, and 
recognized it as a key element in effective learning. 



For metacognition to be effective in learning, students can no longer be 
contained in a secondary role of a passive player in the classroom. 
Students learn to judge their own thoughts, and actions by having 
opportunities to do so with supportive guidance. Students must play an 
active part in monitoring the knowledge they acquire and assessing its 
effectiveness. 
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For some teachers, no longer being the "Sole Source of Information" and 
"Authority Figure" might bring a feeling of insecurity or disorientation. 



If you feel this way, examine your reactions to "letting go" since: 



"The degree to which educators are willing to share control of 
assessment with students will reveal something about the primary 
purposes for doing assessment." 

- Thomas. 1992. p. 49 
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HOW? 



If the only tool you hi /e is a hammer, everything you see will be a nail, 

- Western folk expression 



There is no ONE procedure, practice, test, too!, answer, way, or magic 
wand that can produce the diversity of information needed to accurately 
assess a diverse student body. 

There are, however, some guidelines you can follow to help see how to sort 
through the labyrinth of assessment information already available and also 
still needed. This section will offer suggestions on: 

HOW do you determine if an assessment procedure is 
"promising" or not? 



^ HOW do you assess your own basis and/or bi? s for how you 
judge others? 

HOW do you construct classroom test activities that take into 
account levels of difficulty and processes? 



gf^ HOW do you compile a comprehensive assessment profile of 
the areas that you need to assess? 
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HOW To Determine If An Assessment Procedure Is 
Tromising" or Not 





There are thousands of assessment materials already available that try to 
keep up with the changing realities of learning in today's world, and more 
will keep coming out every day. It can make it difficult to choose which 
ones would be best to meet the needs of you, your program, and your 
students. 



You know you want practices that will make assessment into a diagnostic 
resource for exploring opportunities rather than a label for closing doors. 
To decide which ones will serve that purpose, look at each potential 
strategy with an eye for "excellence", using as the hallmark of a school's 
excellence : 



* An institution where all people help others become something 
more than they ever hoped to be," 

- Lewis, 1986 



L Criteria 



To bring the ideal of "excellence" within the realm of reality, specifically note 
if an assessment practice meets the following criteria: 



1) That the practice is one that can be done with students, not just to 
them. Remember, the word "assess" comes from the Latin meaning 
'to sit beside", and a practice should reflect that idea of not standing 
back and delivering results. Rather, it involves both the instructor 
and the instructed in a communal effort, coming together to 
determine what best works for the student. 
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2) That the practice reflects that knowledge is something the student is 
actively constructing, not passively absorbing and squeezing out. 
Many of the classic "pencil and paper" tests have the student just 
filling in circles for given choices rather than working out how to fill 
out the gaps in his/her own learning. 



3) That the practice evaluates skills within a context for real 
applications. Knowledge is acquired not through the abstract or 
general, but by situating it within something meaningful, providing 
direct evidence of actual performance in school and on the job. 



4) That the practice not only allows for, but also incorporates diversity 
into assessment as part of the total picture needed to see all the 
possibilities of a student. Diversity encompasses cultural, linguistic, 
and open-ended postulations that have no one right or wrong 
interpretation. 

Inherent in diversity is also the concept that not everyone should be 
assessed with the same test. Assessment is a process, not a 
science that has a set equation. Especially with LEP students, some 
tests will measure the language level more than the skill level and 
give a misleading indicator of ability. 



5) That a practice is one that can be used to trace students in terms of 
their progress and development, rather than place them in terms of 
their limitations. This means that the assessment practice should be 
imbedded in the curriculum as an integrative, formative activity 
instead of being an isolated compartment to begin or end learning. 
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Each and every criterion is an essential standard for assessment to meet 
if your students are to reach their potential. Basically, they boil down to 
channeling your choice of strategies by asking yourself three fundamental 
questions: 



1) 


What do you want to see your students 




be able to do as a result of your 




instruction? 


2) 


What improvements, or steps in the 




process are needed to achieve that? 


3) 


How will your students know if and when 




they achieve that? 




HOW to Assess Your Own Basis And/or Bias for How 
You Judge Others 



When you assess a student you are judging something about that person. 
An experienced teacher can call on years spent in the classroom and be 
able to quickly size up a situation or action involving a student. 



But your life is not just spent in a classroom. You bring to your judgements 
experiences accumulated over a lifetime that form a familiar framework for 
you to process information. Your reactions become almost automatic. 



Since the process is activated automatically, you will use it unconsciously 
as your standard for judging situations or people even when unfamiliar 
information is involved. You make culturally-based assumptions which 
could make your judgements inaccurate if you are dealing with a different 
culture or way of doing things. 
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For example, many Americans consider direct eye contact to mean that a 
person is honest and confident. Yet in some cultures, people lower their 
eyes to show respect and humility. 

If such differences are automatically discounted or processed into the 
wrong mental "file", misunderstanding occurs. Carried to an extreme, such 
assessment results in stereotypes. 



The following exercise illustrates the process. 



Catching Your Unconscious •'Assessment* Processes at Work 

Analyze how you tend to naturally "size-up" people or situations by 
following these suggestions: 

1 . Next time your are in an unfamiliar situation, notice how you 
first see or meet a stranger (walking along the street, sitting in 
a restaurant or on the subway). 

2. Notice how you instantly size them up. You will discover that 
you make a whole set of assumptions about the person: 
safe/threat, like/dislike, superior/inferior, sexy/blah, 
interesting/boring, and a number of other conclusions. 

3. If you can recall having done this in the past, then later got to 
know the person, try to remember how long it took to change 
your mental picture of the person. 

Where did you get the criteria for making these 
judgements? 

How did you apply them? 

Can you see the possible impact of such judgments on 
your relationships with people of a different culture? 

(Adapted from Thomas, Grover, Cichon, Bird, & Harns, 1991, p.8.7) 
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HOW to Constaict ClassrcxHn Test Activities That Take Into 
Account Possible Levels of Difficufty and Different Processes 



The focus of this Handbook is to help lead the idea of assessment away 
from the too common 'This is a Test" mind-set, and towards a more 
comprehensive and compassionate process. 

Most manuals advise that this is accomplished best in vocational education 
through the use of performance and authentic assessment strategies, since 
workers need to apply knowledge directly to performing a task and solving 
a problem. 

Most manuals stress that fairer and more accurate assessment is 
accomplished best with LEP students through informal techniques involving 
psychomotor performance tests, teamwork opportunities, and 
multimedia/multilingual means of expression. 

We agree. 

Yet, we also see that there can be important uses for the typical classroom, 
teacher-constructed pencil and paper test. For example: 

• Most occupations require reading and writing skills. Formal written 
tests can help students get incremental feed-back on developing 
those skills and have a sequential record of their development. 

• Test taking skills are also important for some career paths in our 
society. Taking formal, written tests is obviously a possible practice 
for developing skills in dealing with written test situations. 



• There are legal and administrative demands for which formal written 
evidence of a student's knowledge is needed. 



Teacher-made pencil and paper tests can be an efficient use of time 
and an effective two-way communication tool for some types of 
needed facts and knowledge. 
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We have listed some of the valid reasons for using pencil and paper tests. 
If it is clear that this assessment tool should be used only when there is a 
legitimate instructional need for such an approach, then we can start talking 
about how to make it also a fairer and more accurate one. 



Types of Written Tests And Their Con^esponding Levels of Difficufty 



There are 2 basic categories for items that are put into a written test. 

1) Recognition Items: Student has to recognize, or select the correct 

answer from possible answers that are 
provided. Focus is on the product of acquired 
knowledge. 

Examples: Matching, True-False, Multiple 
Choice items 

2) Recall Items: Student supplies the answer. Reveals the 

process of responding with knowledge since 
the student has to know the answer, retrieve it 
from memory, and replicate it in an organized 
manner on paper. 

Examples: Fill-in-the-Blanks, Short Answer, 
Essay questions. 

Since these are categories of items for written tests, obviously the ones 
involving more writing will usually be more difficult, especially for students 
with limited English skills. Therefore, it would be best to use recognition 
items with LEP students, at least until their language proficiency increases 
and they develop more formal test-taking skills. 

«y FIGURE 4-1 (next page) shows the progression of the test items' 
difficulty, based on the type used, the thinking process involved, and 
the language input needed. 
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Adapted from: Lopez-Vpi-dez, J., and Reod (1989) FigUPG 4-1 Page 33 
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General Principles for Writing Test Items 



You know what type of written test you think would be best for your needs 
and your students. Now how do you go about actually putting one 
together, one that will be fair to your students and informative for you? 

Each specific type of test item is unique in construction, just as every 
student is unique in some aspect of the learning process. However, there 
are certain basic points which should be taken into consideration when a 
teacher constructs a pencil and paper test for students. (Adapted from 
Bradley and Friedenberg, 1988) 

1) A test should never be used as a punishment. 

2) Tricky or catch questions should be avoided. 

3) Some teachers find it helpful to put each test item on separate 3 x 
5 cards during the early stages of test construction. The item goes 
on one side of the card and the response on the back. This method 
lets the teacher quickly select or re-arrange items for a particular 
test. It also creates a 'i:est item file" to which the teacher can 
continuously add or delete items. 

4) Test items should be used only after they have been carefully 
checked (better yet, field tested) for clarity and accuracy. 

5) A good test is compoGed of items which emphasize the application 
of lessons learned. 



6) Start with easy materia! and gradually add difficult items. 

7) Place similar types of test items together on the test. 
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8) Directions should be clear and specific with examples provided as 
illustrations. When possible, directions should be provided in 
pertinent languages. 



9) Avoid possibilities of ambiguous answers. There should be a clear 
meaning and response sought by each item. Sufficient evidence 
should be available to support the correct response. 



10) One item should not be based on the response to another, nor 
provide the response clue to another item. 



11) Items should use vocabulary appropriate for the language and 
educational level of the students. 



12) Items should be grammatically correct, use simple sentences and 
direct structure. 



13) Items should be realistic and practical, calling for information that 
students must use. 



14) Minimize or avoid the use of textbook or stereotyped language. 



15) Items should be brief, but not at the expense of clarity. Except for 
essay items, the suggested length is from 3 to 20 words. 

In the case of true-false statements, research has shown that 
sentences with more than 20 words are guessed to be true more 
than 50% of the time. 



16) Items on true-false tests using "open" qualifiers such as "sometimes", 
"often", "several", "as a rule", etc., are generally guessed to be true. 



17) Items on true-false tests using "closed" qualifiers such as "all", 
"always", "never", "exactly", "entirely", etc., are generally guessed to 
be false. 



18) On multiple choice tests, there should be four or five alternatives. 
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HOW to Compile a Comprehensive Assessment Profile 
of the Areas to Assess 





*We must change from a model that picks winners to 
one that will create winners." 

- H. Hodgkinson 

Michigan Employability Skills Task Force 



We have already outlined the areas that need to be developed and 
consequently assessed for full employability possibilities in the changing 
workforce. 

v3t Pages 6-7 of this Chapter. 

That listing was presented as composite highlights, not a definitive 
standard. What you assess will vary with the different needs of your 
students and goals of your program. 

The same open approach applies to HOW to assess each skill area. Here 
we offer some suggested strategies for pulling out and putting together the 
information that will help: 

• your students make appropriate career choices and 
successfully develop them. 

• you to evaluate the need and effectiveness of classroom 
instruction. 

Through cross-references to other chapters in this Handbook and 
descriptions of practices being done by other professionals in the field, we 
present ideas that have worked in settings that are innovating techniques 
to keep up with the times. 
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To try to make the information as clear and less repetitive as possible, we 
mention possible strategies for each assessment area here. If you want 
examples and illustrations of a particular practice, the symbol «3r 

points to the chapters/pages in the Handbook where they are 
explained in more detail. 

You might immediately notice that we do not include in this section a listing 
of which commercial, standardized tests could be used. We do, however, 
devote a separate chapter to that subject. 

& Chapter 10 - Standardized Tests 

In that chapter, we discuss the prevalent and possible usages of 
standardized tests in the assessment process. 

However, you will find that this Handbook is geared more towards 
strategies that can be applied in the classroom to give the student and the 
teacher direct feedback on how needed skills are being developed. 





L Assessing Basic Academic Skills 



The "Basic Academic Skills" are Reading, Writing, and Mathematics, and for 
a LEP student, fair assessment of these areas should be done in both 
English and the native language. 

The inaccuracies that register when LEP students have their academic skills 
tested only in English are obvious. A student with little knowledge of 
English may do poorly on a math test in English, for example, even though 
he/she actually has a strong math background. 
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It is also obvious that assessing the areas in the student's native language 
means that a trained, bilingual staff member is needed. An educator 
familiar with the linguistic/cultural background of the student can offer the 
cultural sensitivity and language nuances needed to break down barriers 
to an accurate assessment. 



A survey of how most Bilingual Vocational Training (BVT) programs assess 
reading comprehension and writing skills of LEP students revealed the 
predominate procedures used are: 

• Cloze tests 

«^ Chapter 6 - CLOZE. 

• Writing Samples 



11. Oral Communication Skills 



There are two important aspects of oral communication: Listening and 
Speaking. Again with a LEP student, these skills should be assessed u^ 
both English and the native language. 

The most common direct assessment tools that BVT programs use for 
assessing LEP students in each area are: 

1. USTENING 

• Dictation 

By Chapter 6 - CLOZE, pages 21-22. 

• Physical responses to given directions, which can be 
assessed through the use of a Rating Scale. 

Chapter 7- Rating Scales. 
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2. SPEAKING 

• Interviews 

Chapter 5 - Inten/iews, pages &-10. 

The development and assessment of oral communication skills of LEP 
students should take into account the complexity of the spoken 
understanding process. For illustration, see 

» graphic on page 1 1 of this Chapter. 

Actual words used carry only 7% of the message. The rest is conveyed 
through vocal skills and body language which can be heavily influenced by 
cultural factors. 

Therefore, for an accurate assessment of a student's oral proficiency, a 
good rule of thumb to follow is: 

Get three oral samples that each contain at least 10-15 minutes of 
the student's language responses. 



111. Occupational Skills, Knowledge, and Aptitudes 



Assessment of these areas with a LEP student explores not only prior job 
knowledge and skills, but also career awareness and job seeking skills in 
this country. 

Occupational skills and knowledge assessment can be largely freed of 
linguistic interference through work samples and performance tasks. These 
specific job-related assessment strategies can be rated according to 
determined criteria. This procedure is explained and examples are given 
in 

Bsr Chapter 7- Rating Scales. 
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Career awareness and job-seeking skills can be explored through pointed 
questions included in an interview with LEP students. For ideas of how this 
can be structured, 

•ap Chapter 5 - Interviews, especially pages 12-13. 



Vocational-related aptitudes are often assessed through standardized tests. 
Matrices of the standardized tests which are most frequently used by 
vocational evaluators to assess this area can be found in 

«3r Chapter 10, Standardized Tests, pages 29-32. 



But even more caution than usual should be applied in the usage of 
standardized tests to assess "Aptitudes". 

Statistical studies have not found a significant relationship between scores 
on these tests and success in vocational programs. 



Perhaps that is because of the misunderstanding of what aptitude really 
means and indicates. It is a more complex, and frequently mis-applied 
process, and not just for LEP students. For a synopsis of the controversy 
that has driven even the SAT tests to change the "Aptitude" part of that 
test's name, see 

Chapter 10, Standardized Tests, page 9. 



For LEP students, the issue is further complicated and misunderstood due 
to language and cultural factors that can obscure or confuse the issue. 
Again, performance tests that actually have a student doing a task can 
more authentically indicate a student's aptitude for picking up something 
quickly. 

Chapter 7 - Rating Scales. 
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An interview that asks the right questions is also a possibility for 
exploring occupational skills, knowledge and aptitudes. 

Chapter 5 - Interviews, especially pages 11-14. 



r 

IV. Flexibilrty/Adaptability 



To explore these areas, that include initiative and motivation, for example, 
the level of acculturation of LEP students is also an important factor to 
consider. How integrated have the individuals become into American 
expectations and functional norms? 

»3r Chapter 10, Standardized Test, page 24, 

gives continuum of Cultural and Linguistic Dimensions which serve as a 
guide for rating your students against the American "norms." 

Asking questions such as those found in 

«3P Chapter 5 - interviews, pages 11-15, 
yields information you can use to evaluate these areas. 

»3r Chapter 8 - Portfolios, page 21 , 

deals with "evidence" the student can collect to demonstrate proficiency 
of these skills. 
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V. Seff-Management and Self-Awareness 

Also known as the level of metacognition, this skill area develops the ability 
to understand and monitor one's own learning process. It involves 
determining a student's interests, learning styles, and organizational skills, 
such as setting goals and meeting deadlines. Portfolios are championed as 
the "King of the Metacognition Hill." 

isr Chapter 8 - Portfolios. 

Other ways to explore and document Interests: 

w Chapter 7 - Rating Scales. Figure 7-10. page 29, 

•ar Chapter 1 0 - Standardized Tests, Figure 1 0-4 on page 30 and 
Figure 10-6 on page 31. 

«3r Chapter 5 - Interviews, pages 12-14. 

Other ways to explore and assess learning styles/preferences: 

• Interviews 

^ Chapter 5 - Interviews, pages 14-15, 

• Observation 

Chapter 10- Standardized Tests, page 24. 
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"Problem solving skills" is considered a fuzzy terrain for many educators 
trying to define it in classroom terms. But it is one of the most important 
skills that employers repeatedly call for and lament the present workforce's 
lack thereof. For a LEP learner, the level of acculturation can give 
indications of being able to define a problem and try ways to resolve it, 
since adjusting to a different culture could definitely be classified as a 
problem. 

Again, Portfolios, as the most encompassing strategy, will demonstrate 
students' level in this skill area. See 

«ap Chapter 8 - Portfolios. 

^ Chapter 9 - Projects. 




Interpersonal skills include teamwork, appropriate behavior in different 
professional/oocial situations, and other very culturally defined activities. 



ra- Chapter 8- Portfolios, page 22- 

Chapter 5- Intemews, pages 25-27. 




Determining special needs that have to be met to insure equity participation 
for all students might mean exploring financial, personal, or linguistic 
situations. For LEP students, obviously language support needs would 
have to be a central focus of this area of assessment. 

Assessment of Basic Skills, page 37-38 of this section. 
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WHERE? 




Some Handbooks You Might Rnd Helpful 



Bradley. C, and Friedenberg, J., Teaching Vocational Education to Limited 
English Proficient Students . (1988). Bloomington, Illinois: Meridian 
Education Corporation. 

Cheng, LL, Assessing Asian Language Performance: Guidelines for 
Evaluating Limited English Proficient Students . (1987). Gaithersburg, MD: 
Aspen Publishers, Inc. 

Cockrum, J., LEP Handbook for Co-Qp Teachers. (1987). Austin, Texas: 
University of Texas at Austin, Extension Instruction and Materials Center. 

Cockrum, J., LEP Handbook for Technology Education Teachers . (1987). 
Austin, Texas: University of Texas at Austin, Extension Instruction and 
Materials Center. 

Cockrum, J., LEP Handbook for Trade and Industrial Teachers . (1987). 
Austin, Texas: University of Texas at Austin, Extension Instruction and 
Materials Center. 

Friedenberg, J., and Bradley, C. A Handbook for Education Agencies: 
Vocational English as a Second Language. (1988). Bloomington, Illinois: 
Meridian Education Corporation. 
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Friedenberg, J., Gordon, R., and Dillman, MA, " Conduct Intake Assessment 
for LEP Vocational Students ", (1988). Columbus, Ohio: National Center for 
Research in Vocational Education. 

Hardy, D.W., Developing Programs for LEP Students In Vocational 
Education: A Handbook for Administrators and Teachers , (1989). Austin, 
Texas: University of Texas at Austin, Extension Instruction and Materials 
Center. 

Hoachlander, E.G., Levesque, K., and Rahn, M., Accountability for 
Vocational Education: A Practitioner's Guide, (1992). Berkeley, California: 
National Center for Research in Vocational Education: University of 
California at Berkeley 

Kapes, J., Parrish, L. and VansickIe,T., Vocational Assessment Handbook . 
(1987). College Station, Texas: Texas A and M University. 

Fradd, S., and Weismantel, M. M. (Eds.), Meeting the Needs of Culturally 
and Linquisticallv Different Students: A Handbook for Educators , (1989). 
Boston, MA: Little, Brown and Co. 

Lehmann, Jean, Sen/ing Limited English Proficient Students in Vocational 
Education , (1989). Fort Collins, Colorado: Colorado State University. 

Leising, J., Lawrence, J., Smith, L, and Enno, W. (Comps), Promising 
Practices for Special Needs Youth in Career-Vocational Education . (1990). 
Rohnert Park, CA: Vocational Education Resource System. 

Lopez-Vaiadez, J, and Reed, T., Building Competencies to Ser^e LEP 
Vocational Students: Inservice Manual , (1989). Des Plaines, Illinois: 
Northwest Educational Cooperative. 

Maxam, S., Informal Assessment: A Handbook for LEAs Sen/ing Special 
Needs Students in Vocational Education . (1985). Columbia, Missouri: 
Missouri LINC, University of Missouri, Columbia. 

Meers, Gary D., Handbook of Vocational Special Needs Education , (1987). 
Gaithersburg, Maryland: Aspen Publications. 
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Mikulin, E., and Patterson, R. Guidelines for Serving Members of Special 
Populations in Vocational Education. (1 992) . Austin, Texas: Texas Education 
Agency. 

Nemko, B., Feichtner, S., Lee, C, Kiesling, M., McGowan, J. and West L, 
Teaching Disadvantaged Students: Successful Strategies in Career- 
Vocational Education . (1990). Rohnert Park, CA: Sonoma State University. 
California Institute of Human Services. 

Norton, R., and Lankard, B., Providing Student Vocational Assessment 
Sen/ices: An Administrator's Guide . (1990). Columbus, Ohio: Ohio State 
University, Center on Education and Training for Employment. 

Sticht, Thomas G., Testing and Assessment in Adult Basic Education and 
ESL Programs . (1990). San Diego, California: Applied Behavioral and 
Cognitive Science, Inc. 

Thomas, Robert J., Bird, L and Grover, J., Serving Vocational Students 
Who Speak English as a Second Language: A Communitv College 
Handbook . (1992). Londonderry, New Hampshire: Capital Consulting 
Corporation 

Thomas, R., Grover, J., Cichon, D., Bird, L, and Hams, C, Job-Related 
Language Training for Limited English Proficient Employees: A Handbook 
for Program Developers . (1991). Washington, D.C.: Development 
Assistance Corporation. 

Thomas, S. and Coleman, N., Vocational Assessment Training Manual . 
(1988). Raleigh, North Carolina: N.C. Department of Public Instruction, 
Division of Vocational Evaluation. 
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Organizations That Are Active in Assessment Issues 



California Assessment Collaborative 

730 Ham'son Street 

San Francisco. CA 94107 

(415) 241-2704 

Designed to support and study classroom, school, or district assessment 
strategies which can be used by teachers. 

Center for Educational Assessment 

403 S 6th Street 

Columbia, MO 6521 1 

Contact: Dr. Steven J, Osterlind, Director 

The center is an office of the College of Education at the University of 
Missouri, Columbia that coordinates psychological research and develops 
standardized testing instruments. Their focus is directed primarily toward 
achievement tests and large-scale programs. They also conduct research 
and create models for alternative assessment. 

Center for Law and Education 

1875 Connecticut Avenue, NW also: 955 Massachusetts Ave. 
Suite 501 Cambridge, MA 02139 

Washington, D.C. 20009 (617) 876-6611 

(202) 986-3000 

A national advocacy support center addressing the education problems of 
low-income students. It has a special project geared to implementation of 
the Perkins Act called Vocational Opportunity for Community and 
Educational Development (VOCED). 

Council of Chief State School Officers 
1 Massachusetts Avenue, NW 
Suite 700 

Washington, D.C. 20001 
(202) 408-8072 

Contacts: Julia Lara; Glenda Partee 

Organization representing heads of State Educational Agencies, with a 
special Resource Center on Educational Equity; sponsors the Workplace 
Readiness Assessment Consortium. 
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Fair Test: National Center for Fair & Open Testing 
342 Broadway 
Cambridge, MA 02139 
(617) 864-4810 

An advocacy group working for fairer testing practices through authentic 
assessment. 

Fort Worth Independent School District 

321 0 W. Lancaster 

Fort Worth. TX 76107 

(817) 878-3807 

Contact: C. Gary Standridge 



National Center for Restructuring Education, Schools, and Teaching 

(NCREST) Four Seasons Project 

Box 110 Teachers College 

Columbia University 

New Yori<, NY 10027 

Contact: Dr. Tenry Baker 

The Four Seasons Project represents a collaboration between the National 
Center for Restructuring Education, Schools, and Teaching; The Coalition 
of Essential Schools; the Foxfire Outreach Network; and Project Zero. The 
primary focus of the three-year project is authentic, performance-based 
assessment practices with an emphasis on increasing student success. 
The project will sponsor summer institutes for teachers from sites around 
the nation and an electronic network dedicated to authentic assessment in 
redesigned schools. 



Quality Education for Minorities (QEM) Network 
1818 N. Street, NW 
Suite 350 

Washington, D.C. 20036 
(202) 65^-1818 

The QEM Network's mission is to unite and strengthen educational 
restructuring efforts to the benefit of minority children, youth, and adults. 
QEM advances minority participation and leadership in the national debate 
on how best to ensure access to a quality education for all individuals. The 
Network disseminates a resource directory which contains information on 
c^uhool-to-work related programs around the country. 
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Sources on Skills Needed in the New Workplace 



American College Testing Program and Council of Chief State School 
Officers, Developing SCANS Assessment Measures: Issues and Options , 
(1992). Iowa City, Iowa: American College Testing Program, 

Bailey, Thomas, Changes in The Nature and Structure of Work: 
Implications for Skill Requirements and Skill Formatio n. (1990), Berkeley, 
California: National Center for Research in Vocational Education. 

Carnevale, A., Gainer, and Metzer, Workplace Basics: The Skills Employers 

Want, (1989). Washington, D.C.: The American Society for Training and 
Development; U.S. Department of Labor, Employment and Training 
Administration. 

Carnevale, Anthony Patrick, America and the New Economy . (1991). 
Washington, D.C.: American Society for Training and Development; U.S. 
Department of Labor, Employment and Training Administration. 

Chisman, Forrest, Spruck Wrigley, and Ewen, ESL and the American 
Dream, (1993). Washington, D.C.: Southport Institute for Policy Analysis. 

Committee on Skills for the American Workforce , America's Choice: High 
Skills or Low Wages! . (1990). Washington, D.C.: National Center on 
Education and the Economy. 

Educational Testing Service Policy Information Center, From School to 
Work, (1990). Princeton, New Jersey: Author. 

Ferry, Cliff, "Employer Occupational Needs Assessment", (1991). Reno, 
Nevada: Nevada State Department of Education. 

Imel, Susan, "Employers' Expectations of Vocational Education", (1990). 
Education Resource Information Center (ERIC), Digest No. 90. 

Johnston, W.B., and Packer, A.E., Workforce 2000 . (1987). Indianapolis, 
Indiana: Hudson institute. 
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Margulies, S., Work Force Literacy Skills for Jobs 2000 . (1990). New York: 
Education Design. 

National Confederation for State Leaders, Making the Connection: 
Coordinating Education and Training for a Skilled Workforce . (1991). 
Washington, D.C.: Author. 

Rosenstock, Larry, "The Walls Come Down: The Overdue Reunification of 
Vocational and Academic Education", Phi Delta Kappan . February, 1991. 

The World Resources Institute, World Resources 1990-1991 . (1990). New 
York: Oxford University Press. 

U.S. Department of Commerce Economics and Statistics Administration, 
1990 Census Profile: Race and Hispanic Origin . (June 1991), No. 2. 

U.S. Department of Labor, Beyond the School Doors: Literacy Needs of 
Job Seekers . (1991). Washington, D.C.: Author. 

U,S. Department of Labor, Secretary's Commission on Achieving Necessary 
Skills, Learning A Living: A Blueprint for High Performance - A SCANS 
Report for America 2000 . (1991). Washington, D.C.: Author. 

U.S. Department of Labor, Secretary's Commission on Achieving Necessary 
Skills, What Work Reguires of Schools - A SCANS Report for America 
2000 . (1992). Washington, D.C.: Author. 

U.S. Department of Labor, Secretary's Commission on Achieving Necessary 
Skills. Teaching SCANS Competencies . (1993). Washington. D.C.: Author. 

The William T. Grant Foundation Commission on Work, Family, and 
Citizenship, The Forgotten Half: Non-College Youth in America . (1988). 
Washington, D.C.: Author. 




Studies and Theories on Changing Assessment 



Association for Supervision and Curriculum Development, A Practical Guid e 
to Alternative Assessment . (1992). Alexandria, VA: Author. 
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Broadfoot, P,, Murphy, R,, and Torrance, H,, Changing Educational 
Assessment: International Perspectives and Trends . (1990), London, New 
York: Routledge, 

California Assessment Collaborative, "An Introduction to the California 
Assessment Collaborative", (1991). Performance Counts . California: 
Author. 

California Assessment Collaborative, Charting the Course:Toward 
Instructionallv Sound Assessment . (1993). San Francisco, CA: Author. 

Center for Law and Education, Problems in Implementing the Perkins Act , 
(1990). Washington, D.C.: Author. 

Council of Chief State School Officers, Resource Center on Educational 
Equity, Summary of Recommendations and Policy Implications for 
Improving Assessment for Limited English Proficient Students . (1991). 
Washington, D.C.: Author. 

Damico, J.S., "Descriptive Assessment of Communicative Ability in LEP 
Students", (1991). In Hamayan & Damico (ed.). Limiting Bias in the 
Assessment of LEP Students . Austin, Texas: Pro-Ed. 

DeAvila, Ed, "Assessment of Language Minority Students: 
Politicaljechnical, Practical, Moral Implications", (1990). Washington, D.C.: 
U.S. Department of Education, Office of Bilingual Education and Minority 
Language Affairs. 

Duran, Richard P., "Assessment and Instruction of at Risk Hispanic 
Students", (1989). The Council for Exceptional Children . Vol. 56, No. 2. 

Fair Test: National Center for Fair and Open Testing. What is Authentic 
Assessment? . (1990). Cambridge, MA: Author. 

Gardner, Hov^ard, "Assessment in Context: The Alternative to Standardized 
Testing", (1992). In Gifford & O'Conner (eds). Changing Assessment . 
Boston: Kluwer Academic Publishers. 

Gardner, Howard, Frames of Mind . (1983). New York, N,Y,: Basic Books. 

Gifford, Bernard, (ed.). Test Policy and Test Performance: Education, 
Language, and Culture . (1989). Boston, Massachusetts: Kluwer Academic 
Publishers, 
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Grfford, B., and O'Connor M., (eds.), thanqinq Assessment: Alternative Views 
of Aptitude, Achievement and Instruction . (1992). Boston, Massachusetts: 
Kluwer Academic Publishers, 

Gould, Stephen J., The Mismeasure of Man , (1981). New York, N.Y.: W.W. 
Norton Publications. 

Guerin, G. and Maier, A., Informal Assessment in Education . (1983), Palo Alto, 
California: Mayfield Publishing Company. 

Hakuta, Kenji, The Mirror of Language: The Debate on Bilinqualism . (1986). 
New York, N.Y.: Basic Books. 

Hamayan, E., and Damico, J., (eds.). Limiting Bias in the Assessment of 
Bilingual Students . (1991). Austin, Texas: Pro-Ed. 

Hoachlander, E.G., "How Should We Use Data to Improve Education?", 
Change Agent Spring, 1991. 

Lara, J., and Hoffman, E., School Success for Limited English Proficient 
Students: The Challenge and State Response . (1990). Washington, D.C.: 
Council of Chief State School Officers. 

Leconte, P., and Boyer-Stephens, "Student Assessment and Support Services: 
The Heart of the Perkins Sequel", The Journal for Vocational Special Needs 
Education . Winter, 1991/Spring, 1992, Vol. 14, No. 213. 

Leconte, P., and Neubert, D.A., "Vocational Education for Special Needs 
Students: Unking Vocational Assessment and Support", Diagnostigue . 1987, 
Vol. 12. 

Lewis, J., Achieving Excellence in Our Schools. (1986). Westbury. New York: 
J.L Wilkerson Company. 

Maeroff, Gene, "Assessing Alternative Assessment', December (1991). Phi 
Delta Kappan . 

Medina, N. and Neill, M., Fallout from the Testing Explosion . (1991). Cambridge, 
Massachusetts: FAIRTEST - The National Center for Fair and Open Testing. 

Mitchell, v., and Russell, E., Exemplan/ Urban Career-Oriented Secondan/ 
School Programs . (1989). Berkeley, California: National Center for 
Research in Vocational Education. 
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Mitchell, Ruth, Testing for Learning, (1992), New York, N.Y.: The Free Press. 

National Commission on Testing and Public Policy, From Gatekeeper to Gateway. 
(1990). Washington, D.C: Auttior. 

Navarrette, C, Wilde, J., Nelson, C, Martinez, R., and Hargett, G., Informal 
Assessment in Educational Evaluation: Implications for Bilingual Education 
Programs. (1990). Washington, D.C: National Clearinghouse for Bilingual 
Education Programs. 

Office of Technology Assessment, Performance Standards for Secondan/ School 
Vocational Education, (1989). Washington, D.C: Autiior. 

Perrone, Vito (Ed.), Expanding Student Assessment, (1991). Alexandria, VA: 
Association for Supervision and Curriculum Development. 

Peterson, M., Vocational Assessment of Special Students for Vocational 
Education: A State of tiie Art Review, (1988). Columbus, Ohio: Ohio State 
University, Center on Education and Training for Employment. 

Phelps, LA., Wermutii, T.R., and Grain, R.L, Vocational Education for Spedal 
Populations:Recommendations for Improving State Policy. (1991), Beri<eley, 
Califomia: National Center for Research in Vocational Education. 

Plihal, J., Rehm, M., and Emst, L The Practice of Equity Access to Treatment in 
Vocational Education in Secondan/ Schools. (1986). St. Paul, Minnesota: 
Minnesota Research and Development Center for Vocational Education. 

Resnick, Lauren, "Assessing The Thinking Curriculum: New Tools for Education 
Reform", (1992). In Gifford and O'Connor (eds.). Changing Assessment: 
Alternative Views of Aptitude. Achievement and Instruction. Boston: Kluwer 
Academic Publishers. 

Schank, Roger C, The Connoisseur's Guide to tiie M ind. (1991). New Yori<, N.Y.: 
Summit Books. 

Simmons, Warren and Resnick, L, "Assessment as tiie Catalyst of School Reform," 
Educational Leadership. February 1993. 

Sosa, Alicia, and Salinas, "Assessment of Language Minority Students: 
Regional Hearing on Hispanics", (1990). San Antonio, Texas: Intercuttural 
Development Research Association. 
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A ssessing Sipedal Populations in Vocational Programs. (1992). Berkeley, CA: 
National Center for Research in Vocational Education. 

Thomas, Ruth, Cognitive Theon/ - Based Teaching and Learning in Vocational 
Education. (1992). Columbus, Ohio: ERIC Clearinghouse. 

Tsang, Chui Lim, "Informal Assessment of Asian Americans: A Cultural and 
Linguistic Mismatch?', (1989). In Gifford (ed.), Test Policy and Test Performance: 
Education, Language, and Culture. Boston, MA: Wuwer Academic Publishers 
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INTERVIEWS 



■ 


IB 


1 


1 



WHAT? 



An interview is frequently the first step that initiates the entire assessment 
procedure. It is a direct manner for gathering important information that will 
determine the direction the rest of the assessment process will go. 




WHAT Informatioa Areas Are Especially Important to 
Explore When Interviewing LEP Students? 



The object of an interview is to get to know a person, to gather background 
information that is needed to make the unknown into an unique human 
being. 

When a student is being interviewed, the focus is on factors that influence 
learning and performance in a classroom. When it is a LEP student, you 
already have a crucial piece of information before the whole process even 
starts. You know that LEP students will be carrying a double learning load: 
confronting new course material while at the same time learning a new 
language and expected way of doing things. 

Therefore, an interview with LEP students should explore areas that give 
hints on how heavy that double load will be for the particular person 
involved, how it can be lightened, and what strengths may be hidden 
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behind language weaknesses. With LEP students, you have a special 
interest in finding out: 

• Educational levels and background in the learner's native 
language and/or country 

• Oral communication level in English 

• Functional skills and acculturation level 

• Career awareness 

• Employment background/skills 

• Vocational interests and goals 

• Learning styles/preferences 




WHAT Can An Interview Be Used For Besides A 
Teacher/Counselor-CcMnducted Placement Routine? 



Typically, interviews are thought of as one of the first steps a counselor or 
person in charge of placement takes to decide what program is most 
appropriate for a student. And that is often the only time that interviews are 
associated with assessment. 

Yet, an interview can provide a format for students to do the asking and 
analyzing, too. It can be a vehicle for students to gather information and 
gain insights liiat make them more aware of their possibilities and 
appreciative of others. On pages 16-29 in the HOW section of this chapter 
we expand the idea of the interview and explain, v;ith examples, that the 
technique can also be a: 

• Self-Assessment and Job Assessment strategy for students^ 

• Technique students can use among themselves to lay the 
basis for teamwork and assessment of communal strengths. 
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WHY? 



Most people have gone through an interview of some kind at some time in 
their life; most people know why inten/iews are done and why they can 
make or break an opportunity. When you go for a job interview, for 
example, you are being assessed on your potential to fulfill a need, to make 
a contribution to collective goals. 



Since an interview is an important, two-way communication process, 
language logically has a tremendous impact on the impressions made and 
information given. Therefore, an interview with a LEP student should be 
conducted by a person familiar with the linguistic/cultural background of the 
LEP candidate. 



When that logic can be made into an actual practice, the interview yields 
answers and valuable insights that no standardized or single-focused 
questions could penetrate. At a multicultural high school in Washington, 
D,C., for example, we met with a bilingual counselor whose 14 years of 
experience enabled him to give teachers more accurate information about 
incoming LEP students he interviewed than what placement test scores 
revealed. 



Why an inten/iew is important and why it should be conducted in a way that 
insures accurate and extensive information are points of fairly common 
knowledge and practice, at least among trained adults leading most 
inten/iews for educational or placement purposes. 
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What is less well known or practiced is the idea of a student determining 
the focus of an interview, making the learner the director of assessment and 
analysis. One possibility of implementing the interview as a student-driven 
strategy is having students "shadow" and interview a worker on the job. In 
the HOW section of this chapter, we explain the steps to do this as piloted 
in the School-Work PLUS project in Indiana. 

Pages 16-22 of this Chapter. 

Here we supply the reasons WHY the interview technique offers unusual 
potential for broadening a student's learning process. 

1) EExperience-Based Learning: By going out into the community to 

interview workers in real world 
situations, the students gain "hands-on" 
experience in addition to classroom 
activity. They have to take initiative, set 
targets, organize their time, work in 
teams, and assess results. 

2) School-to-Work Link: The interview draws attention to skills 

required in the workplace and what 
students need to learn in school. By 
its very nature of structuring a 
student's view of a real workplace, it 
requires collaboration between the 
educational and business communities. 

3) Students as Communicators: Students develop dual communication 

skills doing an interview in this context. 
They learn the formal exchanges with 
adults on the job. and they also have 
to report their findings to other 
students. The "reporting" phase of the 
interview project encourages creativity 
in devising means of communicating 
through videotape, printed materials, or 
other original ways. 

Although dependent on language skills, interviews can offer positive gains 
for LEP students, also. 

«3r Pages 23-24 of this Chapter. 
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HOW? 



During our preliminary work for this Handbook, an earnest and experienced 
vocational education evaluator asked us if we had come across any manual 
that explained how to get all the insights that really good interviewers seem 
to naturally extract from the question-answer process. 

That would truly be the "Silver Bullef. 

But, how can anyone instruct on the acquisition of an "instinct", the sixth 
sense that lets you know how to respond to any unexpected input and 
understand what is really meant? That will come only through experience, 
not from any outline in a handbook. With time and repeated attempts, you 
will learn how you can stretch an interview into the most complete and 
informative part of the assessment process. 



Only you can supply the needed time, practice, and patience. 
Handbook answers in this section is: 



What this 



^ HOW can the inten/iew process yield the optimal amount 
and range of information needed when assessing LEP 
students? 

^ HOW can an interview process be used by students as a 
self assessment strategy with regards to vocational 
possibilities? 

^ HOW can an interview be a classroom technique for the 
student to lay the foundations for teamwork and assess 
communal strengths? 
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HOW to Have An frrt^srview Yield the Optimal Amount 
and Range of Infomnation Needed When Assessing LEP 
Students 



I. Involve Bilingual People and Fonnats. 

The most crucial and optimal piece of the process is to have a person 
familiar with the linguistic/cultural background of the LEP candidate conduct 
the interview. 

Educators may groan and complain that their schools don't have the budget 
to hire multilingual support staff, and in some cases, that is true. But, if you 
have multilingual students in a classroom, you have multilingual resources in 
the community. If budgets don't allow hiring, then you might find a bilingual 
person on a volunteer basis, through religious, parent, or other local 
organizations. Act on the assumption ihat there are people in your community 
that can help open communication with LEP students in your classroom. 

11. Don't Just Play By the Book. 

Sometimes people can be fooled by thinking that because a textbook, a 
school district, a state law, or a federal guideline has produced an official, 
even bilingual, format for questions to be asked of new students, the bases 
are covered. But what has come out of an administrator's office or the 
courthouse or a research paper won't necessarily give you all the answers 
and information you need. 

Rgur^ 5-1 (next page) is the Home Survey Questionnaire Form given 
to al! new students enrolled in the Northside Independent School 
District in San Antonio, Texas, and potentially in need of extra 
support in English. It illustrates how a district's Bilingual Assessment 
Center found the form that was legally prepared and required by 
Texas law did not give them enough information. Relying on only 
the two questions mandated by state regulations for entering 
students resulted in many LEP students falling through the cracks. 

After several staff meetings discussing what was needed and what 
was feasible on a one page form, three more key questions were 
added to the District'?^ Survey. 
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Texas Home Language Survey 



Used in Northside Independent School District, San Antonio, Texas 

Example of Modified Language Information Survey Form 



Student Name: 

Nombre del alumno: 



_ School_ 
Escuela: 



Date of Binh: 



/ 



/ 



Fecha de nacimiento:monihJd^y/y^T 
mes/dCa/ano 

Place of Birth: 



City/Ciudad 



Grade: 
Grado: 



Sex: M_ 
Sexo: 



Has student ever attended school in Northside ISD? No/No 
iHa asistido alguna vez a una escuela de Northside? 



S\2XtlEstado 

Yes/5/ 



ConnWjlPais 



When? 

iCudndo? 

School last attended: ^ _ 

Escuela anterior: Ciudad: 
(Questious one and two are mandated by Texas law.) 



_ What School?, 
6Qu^ escuela? 

City: 



State: 



Estado: 



1. What language is spoken in your home most of the time? 
iQui idioma se habla en su cos a la mayor parte del tiempo? 

2, What language does your child speak most of the time? 
iQui idioma habla su hijo (a) la mayor parte del tiempo? 



(Three additioniJ questions were added for further clarification.) 

3. What language did your child learn when he/she began to talk? 

iQu6 idioma aprendid su hijo (a) cuando comenzd a hablar? 
4* Name the language most often spoken by the adults at home. 

iAnote el idioma que los adults hablan mds en su hogan 
5. What language does your child speak when playing with friends? 

iQud idioma habla su hijo (a) cuando juega con sus amigos? 



Signature of parent/guardian Telephone number 

Firma del padre Iguardidn Niimero de telifono 

(Students in grades 9-12 may sign) Today's Date / / 

(Alumnos en grados 9-12 pueden firmar) Fecha mesldtalano 



FIGURE 5-1 
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III. Expand the Areas That Need to be Explored. 



LEP students conT: from different backgrounds and present a lot of 
"unknowns" not dealt with in interviews typically given to native English 
speaking applicants. Information relevant to program needs and planning 
can only be brought into context by asking different types of questions than 
found on most standard interviews. 



Just as we feel that there is no one assessment strategy that answers all 
the points to bo addressed in the process, we found no one interview form 
to be adequate or extensive enough to include here as an encompassing 
example. 



What we see as being more usefu! is to list the interview information topics 
unique to LEP students. Under each heading, we give sample questions 
that would help to gather the desired data. The list is meant as a 
synthesized guide, not a completed format. Each student is different; 
therefore each interview, or let me "get to know you better" attempt, should 
reflect a difference. 



A. Educational levels and background in the learner's native 
language and country 



Knowing how much schooling a LEP student had before being in American 
classrooms can give you an idea of how much foundation has been laid for 
skills to be able to transfer. You want to find out: 

• Did you go to school in your country? 

• What grade did you finish in school? 

• What did you learn in school? 

• Can you read and write in your language? 

• What sort of things do you like to read in your language? 
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• Do your parents like to read and write in your language? 

• What kind of school work do you do best? 

• What kind of school work gives you the most trouble? 



B. Oral communication level in English 



If the inten/iew is conducted in English, it offers the opportunity to also 
serve as an assessment of the student's oral communication level. 



Figure 5-2 (next page) gives one of the most frequently used 
methods for rating oral proficiency during an interview. 
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Examples of Oral Interview Rating Scale 

1 2 3 4 5 



PRONUNCIATION: 


Foreign ( ) ( ) ( 


) ( ) ( ) Native 


GRAMMAR: 


Inaccurate ( ) ( ) ( 


) ( ) ( ) Accurate 


VOCABULARY: 


Inadequate ( ) ( ) ( 


) ( ) ( ) Adequate 


COMPREHENSION: 


Incomplete ( ) ( ) ( 


) ( ) ( ) Complete 


FLUENCY: 


Halting ( ) ( ) ( 


) 0 0 Smooth 



(developed by the Foreign Service Institute) 



Example of a 5-Level Rating Scale 


Level 1. 


Speech is so halting and fragmentary as to make conversation 
virtually impossible. 


Level 2. 


Usually hesitant; often forced into silence by language 
limitations. 


Level 3. 


Speed and fluency are rather strongly affected by language 
problems. 


Level 4. 


Speed of speech seems to be slightly affected by language 
problems. 


Level 5- 


Speech is fluent and effortless as that of a native speaker. 


(Thomas, Grover, Cichon, Bird, Hams, 1991, p.4.4) 



Figure 5-2 
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C. Functional skills and acculturation level 



Cultural shock can be a paralyzing phenomenon, making a person initially 
confused and unable to fully function. Assimilating and adjusting to a new 
environment is a learning process in itself and can interfere with what has 
to be learned in the classroom. The person in charge of the classroom 
should therefore be interested in asking LEP students: 

• How long have you been in this country? 

© What do you like most about this country? 

What do you like the least about this country? 

What do you find the most confusing thing to understand 
in this country? 

What do you miss the most about your country? 

Do you have many relatives and friends here? 

What language does your family speak among yourselves? 

Which language do you use with friends? 

Did you live in the city or a farm area in your country? 

Do you drive a car? Ride a bicycle? 

Do you have a driver's license? 

Do you know how to use the metro/subway? 

Do you know how to ride the bus? 

Do you understand people on the phone when they speak 
English? 

Can you order food in English in a restaurant? 

When you go to the grocery store, can you read the names 
of food? 

Do you have a bank account? 

Do your parents work? What work do they do? 

Have you been taught English in a classroom situation? 
When ? Where? For how long? 
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D. Job/Career awareness 



For many LEP students, career awareness is part of the acculturation 
process involved in assimilating new cultural information and possibilities. 
A person who has moved to a major U.S. city after growing up in an 
isolated agrarian village in Laos, for example, will be unfamiliar with urban 
vocational possibilities and demands. How close LEP students are to the 
reality of the American job market can be explored through asking: 

• What do you think employers look for when they hire 
someone? 

• What are some of the reasons why people get fired from 
jobs? 

• What would an employer like about you? Not like? 

• Would you like to earn a lot of money in your job? 

• What do you consider to be a lot of money? 

• Is it important to your family that you make a lot of money? 



E. Employment background/skilis 



You want to know the job experience background of al! students enrolled 
in vocational training courses. In the case of LEP students, they might have 
worked with tools which can only be identified by having pictures at the 
inten/iew for illustration purposes. 

• Do you work after school? 

• Do you have responsibility for doing certain chores in your 
home? 
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• Have you ever worked for pay? 

• Do you make money that you can spend? 

• Identify the tools you have used before. (Show pictures). 

• Have you ever used a typewriter? A computer? 



F. Vocational interests and goals 

Earlier in the interview, you explored the career awareness of a LEP student 
to see how close the connection is to the reality of the American workforce. 
You also want to know if there is an interest or possible talent for particular 
areas of knowledge and application of skills. 

• What type of jobs interested you in your country? Did you 
have any experience with them? 

• What types of jobs would your family like you to do? 

• What types of jobs have you seen in this country that are new 
for you? Do they interest you? 

• Why did you enroll (or want to enroll) in this vocational 
education program? 

• What do you hope to do when you finish high school (or 
community college or vocational courses?) 

• What would you like to be doing 5 years after you finish 
school? 

• What things do you enjoy doing? 

• Have you ever made things with your hands? 
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• Do you like to draw and consider that you are Qood at it? 

• Have you ever repaired something that was broken? 

• Have you ever created something, tried to make something 
new that you hadn't done before? 

• Do machines and how they work interest you? 

• Do computers interest you? 

• What tools and/or pieces of equipment do you recognize? 
(Show pictures) 

• Do you enjoy situations in which you have to speak a lot with 
people you don't know? 



G. Learning styles/preferences 



Further on in this Handbook, you will see a chart listing the 15 different 
factors that influence how each individual learns and that determine how 
he/she can be expected to respond in a classroom. Every single point has 
cultural/linguistic dimensions that will help you better understand and 
assess the differences that underlie a person's learning process, 

«ar Chapter 10, Standardized Tests, pages 13-19, and Continuum, 
Rgure 1Q-1, page 25. 

The Continuum gives you a format for pinpointing the degree to which 
linguistic/cultural factors have influenced a student's learning style or 
preference. An Interview can give you general hints and understanding of 
what is involved through questions such as: 

• Do you like to study or work with others? 

• Do you learn best by reading? By listening? By doing? 



ERLC 



Page 14 



Crosspaths Monagomont Systems, tiK. 

105 



Interviews - Chapter 5 



• What makes you feel safe? 

• What makes you feel happy? 

• Do you understand your teachers? 

• Do you understand questions in class? 

• What do you like to do when you are not in school or at 
work? 

• Do you ask your teachers for help when you have problems 
with your work? 

• Where do you prefer to sit in class? In the front? Middle? 
Back? 
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HOW A Student Can Use an Interview As A Self- 
Assessment of Skills and Vcx^ational Possibilities 



Most manuals and many institutions recognize that staff must be attuned to 
a variety of differences when conducting an interview. While heightened 
sensitivity and understanding does make an interview a more 
comprehensive assessment technique, the drive and direction of the 
interview is still focused by the teacher or administrator. 

The idea that an interview strategy can be modified into a total learning 
experience conducted by the student is a promising and more unique 
practice. 

School-Work PLUS 

High school seniors in five Indiana public schools field tested this approach 
by going into their communities' high performance workplaces. The 
students visited factories, offices, and other places of business not only to 
"shadow"' or observe a worker doing his/her job, but also to structure their 
observations through a survey questionnaire procedure. Using a set of 
questions extracted from SCANS and then developed by Pelavin 
Associates. Inc.. in a project called School-Work PLUS , the students were 
able to: 

• define and evaluate the skills needed by different workers 
they interviewed. 

• rate the needed skills in order of their importance, 

• compile the information gathered from the on-site interviews 
into a job analysis for each particular profession, 

«3r Figure 5-3, (pages 17-20) shows the seven step outline that the 
students follow for getting the desired information from the interview 
they are conducting. At the end of the interview, Figure 5-4 
(page 21) shows the five point scale students use for rating the 
importance of the skills they observed being done by their interview 
subject. 
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1. Introduction 

♦ introduce yourself 

♦ explain the project and provide any other background 

♦ explain the purpose of the interview 

2. Background on the Subject and the Job 

Name of Inten/iewer 



Name of Note-Taker_ 
Job Title 



Date of Interview / / 
Name of Jobholder 



Organization or Company Name 
Name 



Address_ 
Phone 



Ask the jobholder how long he/she has been experienced in the job: 

Worked at the job years months 

Supervised the job years months 

3. GoneraJ Job Description 

Ask what the main purpose of the job is, in one sentence if possible. In other words, why does the job exist? 

Ask what the primary or major job duties are - also in one sentence each (there should be three to five of these, although some jobs 

may require more to fully express the purpose of the job). 

Duty 1. 

Duty 2. 

Duty 3. 

Duty 4. 



Duty 5._ 



4. Rating the Skills 

Let the subject look at the Skills Definitions and Scale of Importance at this point. Explain that you want to go through each of the 
twenty-seven skills in order and let the subject rate how important the skill is to the job, using the 5-point Scale of Importance. Point 
out on the scale how there are two criteria to keep in mind how important the skill is and how often it is applied on the job. 

Discuss the skills a bit - pointing out how the first three categories f Foundation Skills') differ from the last four('Functional Skills"), 
and how all three of them differ from the kind of highly specific 'how to' knowledge of any job. 

The note-taker should wrKe in the number that the subject gives to each skill and keep an eye out for the one highly-related skill to 
be chosen In each category for Step No. 6. The Inten/iewer should read the definition of each skill. 

The actual rating of each skill is done by assigning a number from 1 to 5 from the 'Scale of Importance' based on the subject's 
response. 

Figure 5-3 
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Category 1 • Basic Skills 

Reading 

Writing 

Arrthmotic 

Speaking 

Listening 

Categofy 11 - Thinking Skills 

Creative Thinking 

Decision-Making 

Problem-Solving 

Knowing Hew to Learn 

Categoiy lU - Personal Qualities 

Responsibility 

Social Skill 

Self-Management 

Integrity/Honosty 

FUNCTIONAL SKHJLS 
Categofy IV- Resources 

Manages Time 

Manages Money 

Manages Material/Facility Resources 

Manages Human Resources 

Categoiy V - Systems and Technology 

Understands Systems 

Uses Technology 

Categoiy VI - Informational Skills 

Acquires and Evaluates Information 

Organizes and Maintains Information 

Interprets and Communicates Information 

Catogory VII - Interpersonal Skills 

Participates as a Member of a Team 

Teaches Others 

Serves Cllents/Cuiitomers 

Exercises Leadership 

Works with Cultural Diversity 



Rgure S-3 (cent) 

School-Work PLUS: READ America, Inc. 
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5. Solecb'ng a Specific SkiH in Each Categofy 

Aa the Interview progrewes through the rating of the skills ii each successive category, the note taker must select, for further 
exploration, a highly-rated skill from Category IV (Resources) Uo\ i Category VI (Informational Skills) and two more highly rated skills 
from any of the five categories. Make a list of the skills selected and hand it over to the interviewer when the rating of the skills is 
completed. The interviewer then goes on to the next step. 

6. lUustraiing tho Skilto 

Here the interviewer explains to the subject that one skill in each )f four categories has been chosan for further discussion in order 
to provide more of an in depth set of illustrations of what really happens on the job. The interviewee needs to describe one of the 
more important or frequently-performed tasks relating to each of the four skills, and then to describe how each task is performed by 
spelling out the steps he/she takes (each step should include a listing of the materials or equipment that might be used). Five steps 
are usually enough to spell out a task, but if more are needed, continue on another sheet of paper or on the other side of the form. 

Categofy IV. Rmoutcm 

Skill Illustrated 

A. One-Sentence Description of Task 

B. Step by Step description of Task and any Equipment Used. 

Step 1 : 

Equipment: 

Step 2: 



Equipment:_ 

Step 3: 

Equipment:_ 
Step 4: 



Equipment:_ 
Step 5: 



Equipment:_ 



Catogofy VI Infofmaliooal Skiits 

Skill lllustr.ited 

A- One-Sentence Description of Task 



Step by Step Description of Task and any Equipment Used: 
Step 1 : 



Eauipment:_ 
Step 2: 



Equipment:_ 

Stop 3: 

Equipment:_ 
Step 4: 



Equipment:_ 
Step 5: 



Equipment:_ 



Rguro 5-3 (con*1) 
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highly-rated skill from any of the other five categories: 
One-Sentence Description of Task 

Step by Step Description of Task and any Equipment Used: 

Step 1: 

Equipment: 

Step 2: ^ 

Equipment: 

Step 3: 

Equipment: 

Stop 4: 

Equipment: 

Step 5: 

Equipment: 

highly- rated skill from any of the other five categories: 

One-Sentence Description of Task 

Step by Step Description of Task and any Equipment Used: 

Step 1: 

Equipment: 

Step 2: 

Equipment: 

Step 3: 

Equipment: 

Step 4: . 

Equipment: 

Step 5: 

Equipment:_ 

7.WrapAJp 

The noting of 'illustrated tasks' completes the substantive part of the interview. Now is the time to thank the inten/iewee, ask for any 
final comments, perhaps discuss how schooling and/or training might relate (or not relate) to the job, and final^/ to ask permission 
for a follow-up telephone call or two, if that should be necessary. 

Now that you have completed the interview, you should plan as soon as possible to work together to double-check the notes and 
straighten out any confusing references. Also prepare a summary of the interview. You will need this for comparisons to other 
interviews the group has done. 

Figure 5-3 (con*^ 

School-Wofk PLUS ; READ America, Inc. 



Third Choice-Any 

Skill Illustrated 
A. 

B. 



Fourth Choice-Any 

Skill Illustrated 
A. 
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SCALE OF IMPORTANCE 

This is the five point scale that you and your subject will use to 
measure the importance of the twenty-seven skills that you will be 
examining over the course of the interview. The five measures go from 
"Not Important* (with a value of i) to "Extremely Important' (with a 
value of 5). The definition of each rating is also given. You can show 
your interview subjects the "Scale" so they understand how the ratings 
apply to each of the skills when questioned. 

Rating: 

1. Not Important 

This skill is not required for the job. 

2. Somewhat important 

This skill is occasionally required or is needed for m/nor tasks. 

3- Moderately important 

This skill is often required or is necessary for somewhat 
important tasks. 

4. Highly important 

This skill is very often required or is necessary for highly 
important duties. 

5. Extremely important 

Job could not be performed at all without important frequent, 
competent use of this skilK 



Figure 5-4 



SchooUWork PLUS: READ America, Inc. 
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In reproducing the form used by the student in this particular instance, we 
reduced the size of the original document for spacing purposes. Even so, 
we recognize that it looks like a lot, and during the field testing of the 
School-Work PLUS project, some people did question the amount of time 
involved. 

If someone wants to modify the form to better suit their needs and students, 
the seven steps to be adapted are: 

1) Introduction: Student introduces self and explains purpose of site- 
visit and interview. 

2) Background on Subject and Job: Name of company, business of 
company, job title, years of experience on job, and so on. 

3) General Job Description: Student asks what the main purpose of 
the job is, to be answered in one sentence if possible. In other 
words, find out why does the job exist? What are the primary or 
major job duties? These should be listed in 3 to 5 short sentences. 

4) Rating the Skills Involved: A 5 point rating scale is given to rank the 
list of skills highlighted in the SCANS report and as they apply to this 
particular job. The listing of skills are categorized into the seven 
areas SCANS found to be most important for successful functioning 
in a job. 

5) Selecting a Specific Skill in Each Category: The student examines 
four different categories and determines the most highly rated skill 
in each of the four areas. He/She then makes a list of the resulting 
selection and gives the list to the worker being interviewed and 
shadowed. 

6) Illustrating the Skills: The interviewer asks the working professional 
to describe one of the more important or frequently performed tasks 
relating to each of the four skills, and then either demonstrate, or 
spell out step by step, how each task is performed (including the 
equipment that might be used). 

7) Wrap-Up: Thanking the person interviewed, asking for any final 
comments, discuss what schooling/training may relate to the job, etc. 



Page 22 Cros»paths Managomont Systems, Inc. 

113 



Interviews - Chapter 5 




HOW The Structure of the Studerrt Interview/Job 
Analysis Makes ft A Positive Strategy for LEP Students 



The preceding "School-Work PLUS" format example expands the idea of an 
"inten/iew" into an entire learning exercise for the student. It gives 
guidelines for 



• Observing ("Shadowing") 

• Communicating to gather information 

• Evaluating information 

• Communicating that evaluation 

With such an emphasis on communication, how can this be considered a 
promising practice for LEP students? 



Besides the general premise of the whole program being based on sound 
pedagogical principals that meet our established criteria, there are specific 
aspects which can easily incorporate the needs of LEP students and 
present them as developing strengths. 



• The interview conducted by the student is a carefully constructed 
survey tool with a step-by-step format and a clearly delineated scale 
for rating the skills that are observed. Although the students have 
the option of also adding their own questions, they have a basic 
"script" already prepared that can be practiced beforehand. This 
gives LEP students the chance to review for any possible linguistic 
difficulties. 
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• When the students go to a workplace to observe and interview key 
personnel, they go in pairs. Again, the element of teamwork is a 
source of support for all students, but for LEP students, it also can 
give crucial linguistic confidence. 

• Since the students will be making a presentation about what they 
have learned from their evaluative visit to the workplace, visual aids 
accompany the interview process. While one of the students is 
conducting the questionnaire survey, the other is capturing images, 
either through a video recording or snapshots. 



T^.^s element of visual illustration, both during the actual interview and 
for the final product presentation, opens up channels of expression 
for LEP learners, a point that is repeatedly noted in other promising 
practices, such as projects and portfolios. 



Therefore, with one strategy, five important considerations for LEP students 
are incorporated: 



1) A clear, step-by-step format to follow. 



2) A clearly delineated rating scale to organize and prioritize 
information. 



3) A "script" that can be practiced beforehand. 



4) A teamwork approach. 



5) Use of observation and visual aids for gathering and 
presenting information. 
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HOW An Interview Format Can Be A Classroom 
Technique for Students to Lay the Foundations for 
Teamwork and Assess Communal Strengths 



One of the skills most strongly called for in the emerging workplace is 
teamwork. Teamwork is also a supportive strategy for LEP learners to 
make on-going gains with peer guidance. Interviews can be structured into 
a tool through which students can assess each other and themselves in 
terms of differences that can come together and be complimentary wholes. 



EXAMPLE 



Exploratory projects that go into students' portfolios are often done as 
group teamwork. The specific procedure followed at Rindge School of 
Technical Arts, in Cambridge, Massachusetts, as part of its CityWorks® 
curriculum, serves as an example. 



The first step in the instructional/self-assessment exercise leading to forming 
a group is having the students do 'Teamwork Inten/iews". The students 
divide into pairs and follow a questionnaire form with a partner. 



Then the information for all the partners interviewed is pooled, and the 
results presented in a format that highlights any linguistic/cultural differences 
within the group as a positive resource. 



In this manner, through inten/iew5 , the students have gotten to know their 
peers by looking to assess .neir communal strengths and seeing 
differences as additions rather than limitations. 



Figure 5-5 (next page) and Figure 5-6 (page 27) illustrate the format 
used for following this "Teamwork Inten/iew" procedure. 
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This is CityWorks© 
Teammate Interview Exercises 



In the CityWorks course you will be working in teams on a variety of projects. Being on a project 
team is somewhat like being on a sports team. In order for the team to function well, its important 
that you really know your teammates. What are they Kke and what do they like to do? 

Complete the following exercises with a partner. You will be introducing each other to the group 
later so write down everything your partner says that is important to remember. 

1 , Take turns being the interviewer and ask your partner this list of questions. Record their answers 
as you go along. 

What is your name? 



Where do you live? 

What school did you attend there? 

How long have you lived in Cambridge? 



Where else have you lived? (other neighborhoods, cities, countries) 
When is your birthday? (Do you know your astrological sign?) 
What is your favorite music group or kind of music? 
What do you do when you're not in school? 

2. Make up one question on your own to ask your partner. 

3. Now talk to your partner and decide on at least two things that you have in common with each 
other. This could tie anything, for example: where you live; the color of your hair; your favorite 
music; etc. Write down your answers below. 

4. Return to your group. Use the Polaroid camera to take a picture of your partner. The picture 
can show just your partner's face (a portrait), or show their whole body. They can make any kind 
of pose that they wish. 

5. Look at the map of Cambridge and wori< with your partner to try to locate each of your streets 
and houses. Put a pin on the map to show where your partner's house is located. Pin their picture 
to the edge of the map closest to where they live. 

8. Now it's time for introductions. Begin by telling the group your partner's name and pointing out 
where they live on the map. Tell the group how your partner answered the questions. 

7. If there is time at the end of your introductions, each of you try to draw a line on the map that 
shows how you get to the high school from your home. Use a colored pencil to trace along the 
streets which you take to walk, drive or travel by bus on your route to school. 
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Compiling Teammate Interviews 
Group Inventory Form 



Work with your group to fill out the following Group Inventory Form 
We are a team of members; are boys and are girls. Our average age is 

(Number) 

. Of US have grown up in Cambridge. Other places some of us have lived included: 



We speak a total of languages, including: 



If the school had a talent show, our acts would include: 



The most unusual things that we know how to do are: 



of us currently hold jobs as: 



Other jobs we have done in the past include: 



of us regularly help out around the house by 



When WB're not at school or at work, our main interests and hobbies are: 



We know people who know a lot about: 
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Where? 




Places where vocational-oriented interviews 
have been developed 



George Marshall High School 
7731 Leesburg Pike 
Falls Church, VA 22043 
Contact: Norma Colyer 



READAmerica, Inc. 
School«Work PLUS 
Post Office Box 1641 
Shepherdstown. WV 25443 



Rindge School of Technical Arts 
CityWorks© Curriculum Developers 
459 Broadway 
Cambridge. MA 02138 
Contact: David Stephen 
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Handbooks and Manuals With Examples of Interviews 



Callejas, J., The Career Guidance and Counseling of In-School Hispanics: 
Some Practical and Theoretical Considerations, (1985). ERIC Document 
ED#265 269. 

Friedenberg, J., and Bradley, C, A Handbook for Vocational English as a 
Second Language. (1988). Bloomington, Illinois: Meridian Education 
Corporation. 

Friedenberg, J., Gordon, R., and Dillman, M., " Conduct Intake Assessment 
for LEP Vocational Students" . (1988). Columbus, Ohio: National Center for 
Research in Vocational Education. 

Kapes, J., Parrish L, and VanSickle, T., Vocational Assessment Handbook. 
(1987). College Station, Texas: Texas A and M University. 

Lehmann, J., Sen/ing Limited English Proficient Students in Vocational 
Education, (1989). Fort Collins, Colorado: Colorado State University. 

Lopez-Valadez, J., and Reed, T., Building Competencies to Serve LEP 
Vocational Students: Inservice Manual. (1 989). Des Plains, Illinois: Northwest 
Educational Cooperative. 

Mikulin, E., and Patterson, R., Guidelines for Serving Members of Special 
Populations in Vocational Education. (1 992). Austin, Texas: Texas Education 
Agency. 

Thomas, R., Grover, J., Cichon, D., Bird, L, and Hams, C, Job-Related 
Language Training for LEP Employees: A Handbook For Program 
Developers . (1 991). Washington D.C.: Development Assistance Corporation. 
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The Cloze Format and Its Different Variations 



WHAT? 



The Cloze technique is basically an alteration of a written text that calls on 
the reader's comprehension processing to put the text back into its correct 
form. The term "cloze" was first introduced in 1953 by W.L Taylor to 
describe his developed method of deleting words from a text. Taylor got 
the word "cloze" from the gestalt concept of "closure", referring to the 
tendency of people to complete a pattern once they have grasped its 
overall significance. 



The Cloze test format of deleted words denominated by Taylor is still the 
most popular and well known version used today. Well refer to that 
particular format as the "Classic Cloze". In addition, other Cloze formats that 
we explain are: 



♦ Rational Deletion 

♦ Cloze Elide 

♦ C Test 



With each explanation, we give samples to illustrate the particular format. 
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We then address the 

t ADVANTTAGES 

and 

I DISADVANTAGES 

involved in the particular fornnats. The advantages listed for the "Classic 
Cloze" can be applied to the other versions, and can be taken as the 
standard reasoning for using the Cloze technique in general. The 
advantages for the variations of the technique are specific responses that 
researchers made to some of the criticisms of the Classic Cloze. 



CLASSIC CLOZE 



This most common format presents the text with every nth word randomly 
deleted. The reader must call on his/her language storage and 
comprehension to fill in the blanks with the appropriate words that would 
complete the text. 

Sample of Classic Cloze Test 



An auto mechanic is tuning up a car. As she is working 
on the car she discovers that the windshield washers are not 
working; She calls the owner and he 1 the mechanic to 
repair them. 

The 2 must first find the cause ^ the problem. She 
checks her repair 4 and finds these troubleshooting 
suggestions: 

fluid tank. Is it full? 

2. Check Hoses* 6 they damaged, loose, or kinked? Are ^ 
plugged up? 

3. Check electrical connections. Are the 8 loose? 

4. Check wiper/washer switch. Does ^ work? 

I.Mte 2.nMCh«ntc 3.of 4.niw>UB} 5.Cti«ck a.Am 7.thcy a.wifM 0.)t 
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CLASSIC CLOZE 



Advantages 

t Economical to create, 
administer, and score, In terms 
of time and money. 

1 Supported by research to be a 
good indicator of foreign 
language competence. Also its 
validity as a reading 
comprehension gauge for native 
speakers as well as non-native 
ones has been supported by 
research. 

1 Gives real indications as to a 
student's probability for being 
able to handle class materials 
and need for specific instruction. 



] Can be easily changed, and 
thus frequently administered, 
making it an assessment 
technique broad enough to 
cover placement, on-going, and 
exit criteria needs. 

t Yields numerical scores based 
on assessment done with 
instructic" il materials, making it 
a tool that can fulfill 
accountability demands as well 
as determining classroom 
instruction strategies. 



Disadvantages 

I Type of text that is selected and 
the deletion rate that is used will 
make unpredictable changes in 
students' scores. 

1 Performance depends too much 
on structural skills and not 
enough on inferential skills to be 
a real evaluator of reading 
comprehension. 

1 A lengthy text is sometimes 
needed to get enough deletions 
to cover a broad scope of 
language levels. 

1 Confronting total blanks can 
create reactions of frustration 
and anxiety in a student. 

I Random selection of deletions 
does not necessarily mean a 
representative sampling of the 
terms or skills you want to 
assess. 
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RATIONAL DELETION 



This format also deals with deleting words from a text, bui here you select 
the type of words to be deleted, instead of following a random numerical 
formula. Depending on what idea or skill you want to be tested, blank out 
the words central to the concept that you have targeted. 



An auto mechanic is tuning up a car. As she is working, she 
notices that the windshield washers are not working. She calls the 
owner and he tells the 1 to repair them. 

The mechanic must first find the 2 of the problem. She 
checks her repair 3 and finds these trouble shooting repair 
suggestions. 

1) Check fluid 4 > Is it 5 ? 

2) Check 6 Are they damaged, 7 or kinked? Are they 
8 up? 

3) Check wiper/washer 9 Does it work properly? 



Sample of Rational Deletion Test 



1.m«ch«nlc 2.c«um a^manuttt 4.tank S.fuU d.hoMt 7.Ioom 8.p{ugg«d aswitch 



Advantages 



Disadvantages 



Gives greater control over test items, 
and, consequently, the level of 
difficulty, and area that is assessed. 



Being more targeted can also 
limit its assessment scope. 



Ties assessment even closer to 
instruction, as topic covered in class 
can be target of words to be 
deleted. 
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CLOZE-ELIDE 



Sometimes called the Cloze-edit or Intrusive word technique, this version 
alters a given text by inserting extraneous words. The reader edits, or 
elides, the text by crossing out the extra words that don't belong. 

Sample Of Cloze Bide Test 



An auto mechanic is tuning up a car. As she is working on the 
car she discovers that the windshield washers are not working. She 
calls the owner and he asks the mechanic to lean repair them. 

The mechanic must first find the cause of the him problem. She 
checks her repair manual and finds could these trouble shooting 
suggestions. 



is it full? 

Are they damaged, loose, or kinked? Are they 



1 . Check fluid tank. 

2. Check as hoses, 
plugged up? 

3. Check electrical selling connections. Are the wires loose? 

4. Check wiper/washer read switch. Does it work? 

AntwAft: Extraneous wor<te ^ t««n, h[m, coutd, as. Milingr read 



Advantages 

This is the only format of the Cloze 
technique that can be mechanically 
scored, making it even more 
economical and far-reaching in its 
possible application. 

Since words are inserted instead of 
blanked out, the respondent sees a 
normally printed page with no 
imposed perceptual boundaries. 
Having to eliminate rather than 
create can also lower the anxiety 
and frustration factors. 



Disadvantages 

Can result in problematic 
scoring as test taker might cross 
out words which are correct but 
redundant. 

Having extra words inserted 
raises the input of stimuli to be 
processed. This can raise the 
level of confusion and frustration 
for the test taker. 
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C TESTS 



The format of this version is again deleting parts of a text. In the 0 Test, a 
part of a word is eliminated. Every other word in the text has the last half 
of that word as the blank to be filled in by the reader. 



Sample of C Test 



An auto mechanic is tuning up a car. As she is working 
on the car . she discovers that the windshield washers are not 
working. She ca 1 the ow 2 and h^ asks t^ mechanic 
t 5 repair th 6 . 

The mech 7 must f i 8 find t_9. cause o 10 the 
pro b i 1 > She che 12 her rep 13 manual a 14 finds th 15 
troubleshooting suqg e 16 : 

1- Check fi 17 tank, f 18 it fu 19? 

Z Check ho 20 , Are th 21 damaged, lo 22 , or kinked? Are th 23 
plugged u 24 ? 

. 1.ca«t 2.own«f a.h* 4.th« 5.to 6.th«m T.mcchantc 8,find 9.the lO.of ll.problcm la.checKs 13.repajr 
14vand l&itMM latugQMtiooB 17.f1uid 1B.1» 19.fut] 20.hOM« 21.th«y 2^too«• 23.th0y 24.up 



Advantages 



Disadvantages 



Solves the problem of a variable 
deletion rate affecting the students* 
scores. 

Has a more reliable and objective 
scoring since there is just one sure 
answer instead of having to weigh 
possible synonyms. 

Gives more linguistic clues and less 
anxiety/ to the test taker by supplying 
the first part of the word that is 
needed. 



More blanks in text 
could mean more 
frustration for student 
despite having more 
language clues given. 

Has the least research 
to validate it, as the C 
Test format is one of the 
latest to be developed. 



t Uses a variety of short toxts and 
larger number of deletions, 
enhancing the representative nature 
of the language being sampled. 
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WHAT Do Most People Presentfy Use the Ooze For? 



In most present day assessment practices, the Cloze is used as a test of 
the student's language level and needs. 

Several handbooks presently available that touch on the combination 
of vocational education and assessment recommend creating a 
Cloze test as part of the initial language proficiency assessment to 
be done with LEP students. 

A survey of bilingual vocational programs found that the Cloze test 
was the most frequently created tool for assv:;ssing reading 
comprehension levels and placing the students accordingly. 

ESL programs also often create Cloze tests as an initial placement 
instrument or part of the evidence needed for successful exit. 



However, as a result of earlier development work on this Handbook, we 
found that the Cloze format can also be made into a classroom activity 
which the teacher can easily construct and extensively apply. 

^ Pages 21-22 of this chapter provide suggestions for using the Cloze 
format for some alternative teaching/assessment purposes. 



In the instances when the Cloze is used as a test, it is 
important to note that the format tests the appropriateness of 
the level of a certain text or materials. It can *i:est' whether 
certain books could be the effective teaming tools an educator 
wants for a student 
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WHY? 



As we explained in defining the Cloze technique, the term "cloze" was taken 
from the gestalt concept of "closure". There is no exact word in English for 
the German word gestalt, but it refers to a unified whole, a patterned 
configuration that has been placed together. According to the Gestalt 
theory of learning, our brain naturally works to fill in gaps of what we 
perceive as the total picture to be processed. 



The theories developed by the Gestalt school of psychology came out of 
Germany around the turn of the century. Researchers there led the field in 
trying to understand human behavior and thought processes. Today many 
of their theories have been expanded by American cognitive scientists, such 
as Schank in his work with artificial intelligence. 



We have frames, knowledge stmctures ... that enable us to do 
a 'paint by numt)ers* thinking process, filling in the blanks to be 
able to see the whole pra»ss... This phenomenon of lilling in 
the blanks' in your knowledge pervades every aspect of your 
thinking. 

(Schank, 1991, p.57) 
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Since language serves as a vehicle of our thought processes, many 
linguists feel that this mental filMn-the-blank mechanism makes an intricate 
tie between words and thoughts. The Cloze format therefore serves to test 
some of the important skills required in processing language, such as: 



1) The ability to allow for an incomplete or distorted message 

2) The ability to recognize textual redundancy 

3) An awareness of the context of the utterance 

4) The ability to anticipate the language that will follow 



Based on that theory, the Classic Cloze test format was originally developed 
in the early 1950s as a test of the difficulty of reading passages for native 
speakers. During the 1 970s, it received particular attention from researchers 
as an indicator of overall ESL proficiency, and modifications of the basic 
format began to evolve. 

The variations presented in this Handbook are examples of how the format 
has been modified to correct some of the criticisms of how the theory was 
initially applied. Each variation represents a slightly different school of 
thought about the process involved. We summarized some of these critical 
differences in 

ra* "Advantages" and "Disadvantages" listed on pages 3-6 of this 
chapter. 
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HOW? 



In the preceding sections of this chapter, we explained what is involved in 
the various forms of the Cloze techniques, why they are used, and what are 
some of the advantages and disadvantages of each one. 

In this section, you will find 

0^ HOW can Cloze activities indicate students' needs and 
classroom strategies? 

0* HOW to develop and administer each type of the Cloze? 

^ HOW can the Cloze formats be used fof more than just 
placement? 




HOW Cloze Activities Can Indicate Students' Needs 
and Classroom Strategies. 



Once the student has finished the Cloze activity, correct and convert right 
answers into percentages. (For example, if your text has 25 deletions, then 
10 correctly filled in words would be a 40% score. Accept synonyms.) 
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Manuals differ in their interpretations of the levels meant by the resulting 
scores, and most advise that the scores cannot be taken rigidly. To give 
some guideline averages, we compiled the tables given by Guerin ( 1983), 
Vernon (1984), Thomas et.al. (1991), and Lopez-Valadez and Reed (1988). 
Percentages are not exact because some flexibility should be given on both 
the high and low sides. 



0-25% The material will be very hard for the student. 

Student will need considerable ESL assistance 
and reliance on native language. Teachers 
should look for much simpler ways to present 
materials and rely heavily on visual aids. 



30-43% Will need ESL assistance and reliance on native 
language, but use of English can be introduced 
early on. Teachers should initially use simplified 
materials and have students do follow through 
work on team basis, or in groups. 



♦ 44-58% Should be able to handle much of the reading in 

the program with instruction at a reasonable rate. 
Teachers can expect student to follow written 
materials and instructions with the guidance 
usually given to the average native speaker. 



♦ 60% + Should be able to handle the material on own 

with little need for extra clarifications. Teacher can 
assign more independent projects at this level 
and rely on student to take the lead in group 
situations involving reading materials. 
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HOW to Develop and Administer Each Type of the 
Cloze 



' 

CLASSIC CLOZE 



This most common format presents the text with every nth word randomly 
deleted. The reader must call on his/her language storage and 
comprehension to fill in the blanks with the appropriate words that complete 
the text. 



Steps to Be Taken and Factors to Be Considered in Developing Activrty 



1) Select the prose text to be used for the test. This is more difficult 
than it sounds, because the type of text selected can greatly, but 
unpredictably, affect the students' scores. 



To mitigate this factor: 



Look for a text that is as authentic and neutral as possible. This 
means avoiding texts that are heavy with humor, dialogue, colloquial 
expressions, and flowery prose. Also avoid texts that are badly 
written, stylistically marked, or culturally charged. 



2) Decide the total number of deletions that you want the text to have. 
Start deleting after the second sentence. We emphasize that you 
leave the first two sentences intact, because most handbooks 
instruct to leave only the first one before starting the deletions. 
However, confronting a page with blanks can be a confusing, and 
consequently frustrating task. 
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To mitigate this factor: 



Start the text with several complete lines that give an idea of what is 
expected to be developed. Educators that work with native 
speakers who have difficulty with language use the rule of thumb 
that two full opening sentences are needed to start with less 
confusions. We feel that this more complete approach should be the 
modification needed when assessing all students who are limited in 
language proficiency. 



3) Decide the rate you will follow to delete every nth word in the chosen 
text. Again, the choice is one to make with care, since it can affect 
the assessment outcome. 

The common range is from every 5th to every 10th word. Most 
vocational handbooks suggest deleting every fifth word, creating a 
shorter test that is quicker to administer. 

However, studies done with Israeli students (Raatz, 1985) showed 
that making deletions fewer than every seventh word increases the 
frustration and irritation that students can experience with the test. 
Going as high as every 10th word creates another problem by 
requiring too lengthy a text. 



To mitigate this factor: 



Try the rate of deleting every 7th or 8th word. Do not include in the 
deletions proper nouns or numbers. 
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4) Show the students a sample text of what is expected of them. 
Take them through a trial, very short version, having them 
follow on their own papers what you are doing and 
explaining. When they are doing the actual test on their own, 
note how long it takes them, how much they might struggle 
with the exercise. 
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RATIONAL DELETION 



This format also deals with deleting words from a text, but here you must 
select the type of words to be deleted, instead of following a random 
numerical formula. Depending on what idea or skill you want to be tested, 
you blank out the words central to the concept that you have targeted. 



Steps to Be Taken and Factors to Be Considered in Developing Acth/rtv 



1) Select the prose text to be used for the test. This is more difficult 
than it sounds, because the type of text selected can greatly, but 
unpredictably, affect the students' scores. 



Look for a text that is as authentic and neutral as possible. This 
means avoiding texts that are heavy with humor, dialogue, colloquial 
expressions, and flowery prose. Also avoid ones that are badly 
written, stylistically marked, or culturally charged. 



2) Decide the total number of deletions that you want the text to have. 
Start deleting after the second sentence. We emphasize that you 
leave the first two sentences intact, because most handbooks 
instruct to leave only the first one before starting the deletions. 
However, confronting a page with blanks can be a confusing, and 
consequently frustrating task. 




To mitigate this factor: 



BEST COPY AVAILABLE 
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To mitigate this factor: 



Start the text with several complete lines that give an idea of what is 
expected to be developed. Educators that work with native 
speakers who have difficulty with language use the rule of thumb 
that two full opening sentences are needed to start with less 
confusions. We feel that this more complete approach should be the 
modification needed when assessing all students who are limited in 
language proficiency. 



3) The words that you delete depend on what topic or particular skill 
you want assessed. The VESL teacher might target the words that 
correspond to a grammatical structure being taught in the class. 
The carpentry teacher might focus on the words in the text that 
describe the tools he has just shown students to use. 



4) Show the students a sample of what is expected of them. Take 
them through a trial, very short version, having them follow on their 
own papers what you are doing and explaining. When they are 
doing the actual test on their own, note how long it takes them, how 
much they might struggle with the exercise. 
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CLOZE ELIDE 



Sometimes called the Cloze-edit or intmsive word technique, this version 
alters a given text by inserting extraneous words. The reader edits, or 
elides, the text by crossing out the extra words that don't belong. 



Steps to Be Taken and Factors to Be Considered in Developing Activit v 



1) Select the prose text to be used for the test. This is more difficult 
than it sounds, because the type of text selected can greatly, but 
unpredictably, affect the students' scores. 



To mitigate this factor: 



Look for a text that is as authentic and neutral as possible. This 
means avoiding texts that are heavy with humor, dialogue, colloquial 
expressions, and flowery prose. Also avoid ones that are badly 
written, stylistically marked, or culturally charged. 



2) Decide the total number of insertions that you want the text to have. 
Start inserting after the second sentence. We emphasize that you 
leave the first two sentences intact, because most handbooks 
instruct to leave only the first one before starting the insertions. 
However, confronting an altered text can be a confusing, and 
consequently frustrating task. 
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To mitigate this factor: 



Start the text with several intact lines that give an idea of what is 
expected to be developed. Educators that work with native 
speakers who have difficulty with language use the rule of thumb 
that two full opening sentences are needed to start with less 
confusions. We feel that this more complete approach should be the 
modification needed when assessing all students who are limited in 
language proficiency. 



3) Use a set of random numbers from seven to fourteen to count off 
words in the text. Follow the sequence of random numbers to count 
and then insert unrelated words into text accordingly. 

Leave the first part of each sentence intact; at least 7-10 words 
remain untouched before new words are inserted, if a word is 
sequenced to be inserted after a proper noun, put the inserted word 
back one space in the counting. 



4) Show the students a sample of what is expected of them. Take 
them through a trial, very short version, having them follow on their 
own papers what you are doing and explaining. When they are 
doing the actual test on their own, note how long it takes them, how 
much they might struggle with the exercise. 
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C TESTS 



The format of this version is again that of deleting parts of a text. In the C 
Test, a part of a word is eliminated. Every other word in the text has the 
last half of that word as the blank to be filled in by the reader. 



Steps to Be Taken and Factors to Be Considered in Developing Activftv 



1) Select the prose text to be used for the test. This is more difficult 
than it sounds, because the type of text selected can greatly, but 
unpredictably, affect the students* scores. 



To mitigate this factor: 



Look for a text that is as authentic and neutral as possible. This 
means avoiding texts that are heavy with humor, dialogue, colloquial 
expressions, and flowery prose. Also avoid ones that are badly 
written, stylistically marked, or culturally charged. 



2) Start deleting after the second sentence. We emphasize that you 
leave the first two sentences intact, because most handbooks 
instruct to leave only the first one before starting the deletions. 
However, confronting a page with blanks can be a confusing, and 
consequently frustrating task. 
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To mitigate this factor: 



Start the text with several compiete lines that give an idea of what is 
expected to be developed. Educators that work with native 
speakers who have difficulty with language use the rule of thumb 
that two full opening sentences are needed to start with less 
confusions. We feel that this more complete approach should be the 
modification needed when assessing all students who are limited in 
language proficiency. 



3) Since there are many more deletions made using the C Test format, 
the texts can be very short, and more than one used (some linguists 
suggest using as many as 5 different texts with 25 deletions in each). 
Using multiple texts within one test can broaden the areas that can 
be assessed at one time. 



4) Delete the last half of every second word in the text. Words of one 
letter are ignored for counting purposes. With the words that have 
an uneven number of letters, delete the extra letter. 



5) Show the students a sample of what is expected of them. Take 
them through a trial, very short version, having them follow on their 
own papers what you are doing and explaining. When they are 
doing the actual test on their own, note how long it takes them, how 
much they might struggle with the exercise. 
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HOW the Cloze Format Can Be Used for More Than 
Just Placement Purposes 



Too often the Cloze technique is reduced to a passive pencil and paper 
task which the student confronts "cold turkey" and gets equally cold results. 



However, the Ck c,e format presents possibilities that can extend beyond its 
most common current use for placement purposes. It can be used as a 
formative, on-going assessment tool that is part of the instructional process. 



We have shown modified versions of the Cloze format to illustrate that a 
basic tool can take several different forms depending on the different 
factors which are taken into account. By adding other steps, the Cloze 
technique can be modified further for the benefit of both student and 
teacher. A classroom teacher can structure a Cloze activity to gain 
assessment insights into the needs and levels of students in the class who 
might respond better to multi-modality means of communication. 



1) After the initial diagnostic administration of the text with deletions, 
give the full, original text to the student. Have him/her go through 
and underline the parts he/she doesn't understand. 

This takes the student into the assessment process on a 
metacognitive level, giving the student the opportunity to assess 
what he/she does or does not know. 

It also lays the foundation for taking the process even further and 
into the oral area. 



2) After the student has read through and marked the full text, he/she is 
then asked to read it aloud (Here it is stressed that a teacher should 
never give any student, much less someone with little linguistic 
confidence, something totally unfamiliar to read aloud). The student 
then discusses it ("discuss" is important, since the student also has 
questions, not just the usual situation of being pressed for answers). 
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The Foreign Service Rating Scale for Oral Proficiency, or the Slalom 
Scale, are methods that can be used to determine verbal fluency as 
the student discusses the text. Those scales are given as guidelines 
in the Handbook's chapter on Interviews. 

rar Chapter 5 - interviews, page 1 0. 



3) The next step involves applying a technique frequently used in any 
learning situation dealing with language development interference. 
To carry the modifications of the Cloze into a more complete 
procedure, the student is again given the original deleted text that 
was the introduction to the whole process. 

Now that the student has worked with the complete text using 
different modalities, he/she tries again to fill in the blanks. 

This gives an idea of short term memory storage as well as a way of 
seeing how deep the confusions are and how many repeated 
exposures to new stimuli might be needed for the student to 
incorporate the information. 
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WHERE? 




Handbooks That Contain Suggestions for Constructing 
and Using the Cloze Test 



Cordova, R.M. and Phelps, I.A., Identification and Assessment of LEPs in 
Vocational Education Programs: A Handbook . (1982). Champaign, Illinois: 
Office of Career Development for Special Populations, University of Illinois. 

Friedenberg, J., Gordon, R., and Dillman, M., "Conduct Intake Assessment 
for LEP Vocational Students" , (1988). Columbus, Ohio: National Center for 
Research in Vocational Education. 

Lopez-Valadez, J. and Reed, T., Building Competencies to Serve LEP 
Vocational Students: Inservice Manual , (1989). Washington, D.C.: Northwest 
Educational Cooperative. 

Thomas, R., Grover, J., Cichon, D., Bird, L, and Harns, C, Job Related 
Language Training for LEP Employees: A Handbook for Program 
Developers ,(1 991). Washington, D.C.: Development Assistance Corporation. 

Vernon, Valerie, J., Vocational Education for Limited English Speaking 
Students , (1 984) . Philadelphia: Temple University, Department of Vocational 
Education. 



BEST COPY AVAILABLE 



CroMpoths Managofnont Systoms, inc. 

143 



Page 23 



Cloze - Chapter 6 




Additional Written Sources Containing Research on 
Ctoze Tests 



Culhane, T., (ed.), Practice and Problems in Language Testing . (Sept. 
1981). Essex University, Colchester, England: Dept. of Language and 
Linguistics. 

Culhane, T, (ed.), Practice and Problems in Language Testing . (Sept. 
1984). Essex University, Colchester, England: Dept. of Language and 
Linguistics. 

Fotos, S., "The Cloze Test as an Integrative Measure of EFL Proficiency: A 
Substitute for Essays on College Entrance Examinations?", (1991). In 
Language Learning. Volume 41:3, pp. 313-336. 

Guerin, G., and Maier, A., Informal Assessment in Education , (1983). Palo 
Alto, California: Mayfield Publishing Company. 

Manning, W., "Development of Cloze-Elide Tests of English as a Second 
Language", In Test of English as a Foreign Language (TOEFL): Research 
Reports. Educational Testing Sen/ice, Report 23, April 1987. 

Piper, A., "A Comparison of the Cloze and C-Test as Placement Test Items", 
(1983). British Journal of Language Teaching. Vol. 21, No. 1, Spring 1983. 

Raatz, U. and Klien-Braley, C, "The C-Test: A Modification of the Cloze 
Procedure", (1981). In Culhane (ed.), Practice and Problems in Language 
Testing. Colchester, England. 

Raatz, U., "The Factorial Validity of C-Tests". In Culhane (ed.), Practice and 
Problems in Language Testing. (1984). Colchester, England. 

Raatz, U., "Tests of Reduced Redundancy-The C Test, A Practical Example", 
Language Testing in School: AF in LA Yearbook. (1985). No. 41. 

Schank, Roger C, The Connoisseurs's Guide to the Mind , (1991). New 
York: Summit Books. 
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Siefer, N., Language Assessment for Limited English Speakers in Vocational 
Programs . (1981). Mesa, Arizona: Mesa Community College. 

Sussmilch, E. "Language Testing with Immigrant Children", (1984). [Papers 
from the International Symposium on Language Testing (7th, Colchester. 
England), 1984] 

Weir, C, Communicative Language Testing. (1990). United Kingdom: 
Prentice Hall International Press. 
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RATING SCALES 




WHAT? 



The Rating Scale is one of the simplest forms of assessment through 
observation. With a format that is essentially a listing of what you want to 
know. Rating Scales give a clear procedure for judging a student's 
performance relative to specific criteria. 



Some people use "Rating Scales" and "Checklists" interchangeably, as if 
they meant the same thing. They both are based on one basic premise, 
but they are really two different approaches. 



A Checklist is a much simpler form, it is a guide to follow for 
quickly checking whether something has been done or not, 
giving the basic 'Ves-no", bottom line information. Obviously, 
this format is not a very sensitive instrument, but it can be an 
efficient one in certain situations. 



A Rating Scale indicates how well a task or behavior is being 
done on a continuum of possible I^^vels. It marks at what 
stage or to what degree something is being done, giving 
depth to the Information that can be gathered through this 
technique. 



We provide one example of a Checklist in the HOW section of this chapter 
(page 37). However, our principal focus and use of the term, Rating Scale, 
Is to explain the use of a continuum in determining different levels of the 
factors to be assessed. 
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WHAT Are the ADVANTAGES and DISADVANTAGES 
of Using A Rating Scale? 



T ADVANTAGES 



• A way to get direct, meaningful information. You observe and 
evaluate what you are seeing. Especially for LEP students, doing 
rather than explaining can convey a more accurate picture of 
abilities. 

• Easy to construct and administer. 

• Provides dear procedures and resutts. 

• Can be frequently and unobtrusively administered. 

• Can be used for entry, on-going, and/or exit assessment. 



I DISADVANTAGES 



• Can be very subjective. This "distractor" can be somewhat controlled 
by establishing clear-cut criteria, but still, one person might judge a 
situation differently than another. 

• Lack of validity, in statistical, technical sense of the word. 

• Has a high *iickleness of the momenf factor. Responses can vary 
according to moods, interactions of particular people, or other 
temporary and difficult to control influences. As one vocational 
curriculum developer in Ohio explained to us: "There's always the 
risk that the performer will remind the evaluator of his nerdy brother- 
in-law and get flunked automatically." 



— 
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WHAT Type of Scales Are Some Educators Using To 
Rate Vocational Competendes and Behaviors? 



Rating Scales can be applied to a wide variety of situations and in flexible 
formats. Teachers, mentors, vocational trainers and even com.puterized 
systems are applying the concept of Rating Scales to evaluate work 
competencies, habits and behaviors. Some examples from the field of 
possibilities we thought to be most representative of the spectrum are 
presented here. 




When a Rating Scale is used to assess vocational competencies, there are 
three possible scopes of evaluation. We give examples of each, starting 
with the most basic and going to the most extensive as follows: 

1) Specific Tasl< Performance 

2) Coursework Competencies 

3) Career Competencies Across Integrated Curriculum 

1) Specific Task Performance. A Rating Scale can show how close a 
student is to mastering a particular performance objective. The 
steps that are involved in carrying to completion a specific task are 
outlined and each part is rated for the level of competency. 

ra- Figure 7-1 (next page) is from a program for training LEP employees 
to be more proficient with the telephone. It shows the message that 
was practiced, the response form to be completed and the Rating 
Scale that was used to test the competency with which the student 
did the exercise. 
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Example of a Simulated Telephone Call Requiring a Written Message and Response Fomi 
Student Is expected to complete. 



operator: 

Caller: 

operator: 

Caller: 

operator: 
Caller: 
Operator: 
Caller: 

Operator: 
Caller: 



Operator: 



Hello, Adurt ESL 
May 1 speak to Timmio Walker? 

Tm sorry but she is not available, she 

won't be In till tomorrow. 
Could you tell her that Bob Thomas from 
DAC called. 

What does DAC stand for? 
Development Assistance Corporation. 

Was that 'Systems Corporation'? 
No. 'A S S 1 S T A N C E". 
The number is (603) 742-6300 

Is thero an extension? 
No. Please tell her that it is very important 
that 1 speak with her tomorrow about the 
training I will be doing at the company next 
week. 

i will. 



TO 



IMPORTANT MESSAGE 



DATE 



TIME 



A.M. 
P.M. 



M_ 
OF 



WHILE YOU WERE OLTT 



Area Coda 



TELEPHONED 




PLEASE CALL 




CALLED TO SEE YOU 




WILL CAU AGAIN 




WANTS TO SEE YOU 




URGENT 




RETURNED YOUR CAU 









MESSAGE 



OPERATOR 



Perfonnance Test for Telephone Role Play 



TASK 

PERFORMANCE 
OBJECTIVE 

RELATED 

COMMUNICATIVE 

COMPETENCIES 



Record telephone messages 

In a classroom role play, record all the information for a telephone message on a 
'While You Were OuT message pad at the level percent of full competency. 



• Understand speech over the phone. 

• Report/write factual information. 

• Ask for clarification. 



• Ask for additional/complete information. 

• Ask for/rocord the spelling of names. 

• Read, understand and use message pad. 



FTEM 


CfUTERIA FOR FUU CREDfT 


0 


1 


2 


NOTES 


I.To 


first & last names 










2. Date 


month, day (& year) 










3. Time 


hour & minute (circle AM or PM) 










4. M 


complete M (Mr., Mrs., Ms.), first & last names 










5. of 


name of organization 










6. Area code & exchange 


area code & exchange, extension if given 










7. Call box check 


check appropriate box(es) 










8. Message, content 


include all information given 










9. Message, mechanics 


correct spelling & grammar clear & legible 










10. Operator 


first & last names 








TOTAL SCORE 


SCORE 

Ratina: 0 = missina 1 = oartial 2 = cAmnletA 











Criteria for Competency: 20 = full competency 1&-19 = minimal competency below 16 = below competency 
'^^Q'^Qo Date Attempt Trainer's Signature 



Source: Thomas, Grover, Clchon, Bird, & Hams 1991 
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2) Coursework Competencies within Vocationa! Education Curriculum 
Rating Scales can be used to cover the skills students are expected 
to develop during the entire course of study in a vocational class. 
It looks at the multitude of competencies that pertain to a whole 
subject area of learning. 



•sr Figure 7-2 (next page), shows a Rating Scale used for evaluating the 
competencies to be developed by students taking an Auto 
Mechanic's Helper Course. 
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AUrO MECHANJCS HELPER COURSE COMPETHNaES 

Rating Scale: 

4 - Skilled - Car work independently (no supervision) 

3 • Modemtely skilled - Can perform job completely wKh limited supervision 

2 • Limited skill - Requires instruction and close supervision 

1 - No exposure/No experience or knowledge in this area 



SKILL(S) TO BE RATED 


4 


3 


2 


1 


SHOPSAFEFf 


1. Demonstrate personal safety on job. 










2. Use protective and appropriate clothing (shoes, goggles, etc.). 










3. Recognize and eliminate hazardous conditions to avoid accidents. 










4. Use chemicals &nd cleaners safely. 










CARE AND USE OF HAND TOOLS 


1 . Identify hand tools by name. 










2. Identify hand tools by purpose (use). 










3. Use hand tools safe!/. 










CARE AND USE OF MECHANICAL EQUIPMENT 


1. Identify mechanical equipment by name. 










2. Identify mechanical equipment by purpose. 










3. Use mechanical equipment properly and safely. 










SERVICES AND PROCEDURES 


1. Use a flat rate manual. 










2. Use a service manual. 










3. Fill out a shop ticket 










4. Lock up parts in parts book. 










BRAKES 


1. Identify all moving parts of brake system. 


1 








2. Differentiate between disc and drum brake systems. 










3. Inspect brake system. 










4. Use chemicals and cleaners safely. 










5. Rebuild wheel cylinder. 










6. Repack wheel bearings. 










7. Replace oil seals. 










8. Remove and replace brake pads and shoes. 










9. Turn drums and rotors on lathe. 










lO.Adjust and bleed brake system. 











Source: Lop«z-Varadez, J. and Bead. (1989) Figure 7-2 
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3) Career Competencies Across Integrated Curriculum. This is an 
ambitious but possible undertaking. It is so extensive that perhaps 
it is more feasible for administrators or counselors than teachers, at 
least for organizational reasons. A list can be compiled of the 
vocational competencies needed to obtain and retain employment 
after completion of schooling. The teachers of all the courses 
involved in the program rate the students on the degree of job 
readiness displayed with regards to the list of needed skills. 



«3r Figure 7-3 (pages 8-10) shows the inside contents of the "Career 
Portfolio" developed by the University of North Texas. Through a 
grant from the Texas Education Agency, a study was made of 
vocational related competencies needed by special needs students 
to obtain and retain employment upon graduation from high school. 
The consequent Rating Scale is illustrated in reduced format here. 
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Rating Vocational - Related Competencies 



DiRECnONS 

Evaluate the «tudent, U3ing the Rating Scale on the right Chock the appropriate number to indicate the degree of competency. The 
rating for each of the tasks should reflect job readiness rather than the grade given in the class. 

Rating Scale 

N * No exposure • no experience or knowledge in this area 

1 • Student has been introduced to the competency 

2 • Student has some ability to perform the competency; requires frequoni supervision 

3 - Student can perform the competency with Smiled supervision 

4 - Student can perform the competency iruiependentiy wKh no supervision; job ready 



EMPLOYABILnY SKILJLS (Competencios that wili enable an individuaJ to obtain and retain a job.) 

The student can : SCALE 

1. Establish realistic career/goals N 1 2 3 4 

2. Display a posrtive attrtude toward work (work ethic) N 1 2 3 4 

3. Demonstrate a good record of attendance N 1 2 3 4 

4. Display punctuality at school, work and following breaks N 1 2 3 4 

5. Display pride in work N 1 2 3 4 

6. Demonstrate honesty N 1 2 3 4 

7. Demonstrate dependability N 1 2 3 4 

8. Observe and follow classroom/work rules and regulation N 1 2 3 4 

9. Display InKiative (e.g., begin work wKhout being asked, assume 

additional responsibility, help others voluntarily) N 1 2 3 4 

10. Work at a constant pace. N 1 2 3 4 

1 1 . Manage time appropriately N 1 2 3 4 

12. Demonstrate Work stability (remains on the job/task until completed) N 1 2 3 4 

13. Work efficiently under pressure or within time limits N 1 2 3 4 

14. Keep work area clean N 1 2 3 4 

15. Display respect for other people N 1 2 3 4 

16. Show respect for property of others N 1 2 3 4 

17. Seek help when needed N 1 2 3 4 

18. React appropriately to constructive crrticism N 1 2 3 4 

19. Accept praise appropriately N 1 2 3 4 

20. Assume responsibility for own actions/behaviors N 1 2 3 4 

21 . Demonstrate appropriate reactions to own mistakes 

(e.g., acceptance, correction) N 1 2 3 4 

22. Demonstrate appropriate problem-solving skills 
(e.g., identify problem, list possible solutions, 

select a solution, evaluate results) N 1 2 3 4 

23. Demonstrate willingness to loarn new skills/Information N 1 2 3 4 

24. Demonstrate adaptability to changing sKuatlons N 1 2 3 4 

25. Follow safety regulations N 1 2 3 4 

26. Respond appropriately to classroom and/or job related emergencies N 1 2 3 4 

27. Practice good hygiene/grooming N 1 2 3 4 

28. Dress appropriately for work/specific job N 1 2 3 4 

29. Correctly complete a job application N 1 2 3 4 

30. Demonstrate appropriate job interviewing skills N 1 2 3 4 

31. Demonstrate the ability to complete a job resume N 1 2 3 4 



nGURE7-3 



Source: 
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GENERAUZABLE SWULS (Competencioe whtch are basic, riecessaiy and transferable, or common, across vocationaJ programs and 
job«. Th«M competoncMs are commonly believed to be rwcessary for success in vocationaJ programs and emptoyment) 
(Greeoan,1966) 



MATHEMATICS SKILLS SCALE 
Whole Numbers 

1. Read, write, and count single digit whole numbers N 1 2 3 4 

2. Read, write, and count muKiple digit whole numbers N 1 2 3 4 

3. Add and subtract single digit whole numbers N 1 2 3 4 

4. Add and subtract multiple digit whole numbers N 1 2 3 4 

5. MuKiply and divide single digit whole numbers N 1 2 3 4 

6. Muttiply and divide multiple digit whole numbers N 1 2 3 4 

7. Use addition, subtraction, muKiplication, and division to solve word 

problems with single and multiple digit whole numbers. N 1 2 3 4 

Fractione 

8. Read and write common fractions N 1 2 3 4 

9. Add and subtract common fractions N 1 2 3 4 

10. Multiply and divide common fractions N 1 2 3 4 

11. Solve word problems with common fractions N 1 2 3 4 

Decimals 

12. Carry out arithmetic computations involving dollars and cents N 1 2 3 4 

13. Read and write decimals in one or more places N 1 2 3 4 

14. Round off decimals to one or more places N 1 2 3 4 

15. Add and subtract decimals in one or more places N 1 2 3 4 

16. Solve word problems with decimals in one or more places N 1 2 3 4 

Measureinent and Calculation 

17. Use a calculator to perform basic arithmetic operations to solve 

problems N 1 2 3 4 

COMMUNICATIONS SKILLS 
Words and Meanings 

18. Use plural words appropriately in writing and speaking 

1 9. Use appropriate contractions and shortened forms of words by using an 
apostrophe in writing and speaking 

20. Use appropriate abbreviations of words in writing and speaking 

21. Use appropriate word choices in writing and speaking 

Beading 

22. Restate or paraphreise a reading passage to confirm understanding of what 
was read 

23. Read and understand forms 

24. Read and understand short notes, memos, and letters 

25. Understand the meanings of words in sentences 

26. Use a standard dictionary to obtain the meaning, pronunciation, 
and spelling of words 

27. Use the telephone book to look up names, telephone numbers, and other 
information to make local and long distance calls 



N 1 2 3 4 

N 1 2 3 4 
N 1 2 3 4 
N 1 2 3 4 



N 1 2 3 4 

N 1 2 34 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 



Writing 

28. Compose logical and understandable written correspondence, memos, 

short notes, or reports N 1 2 3 4 

29. Write understandable statements, phrases or sentences to accurately 

fill out forms N 1 2 3 4 



Speaking 

30. Speak fluently with individuals or groups 

31. Pronounce words correctly 

32. Spoak effectively using appropriate behaviors such as eye contact, 
posture, and gestures 

Listening 

33. Restate or paraphrase a conversation to confirm what was said 

34. Ask appropriate questions to clarify another's written or 
oral communication 



N 1 2 3 4 
N 1 2 3 4 

N 1 2 3 4 



N 1 2 3 4 
N 1 2 34 
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JOB SPECIFIC SKILLS (Competencies that will enable an individual to work effectively on a specific job) 

Note: In addition to the competencies listed below, a competency profile of the specific job trtle should bo 
obtained from the vocational instructor, or a job analysis should be conducted with the employer. 



The student can: 



SCALE 



1 . Seek clarification of unclear directions or assignments 

2. Use and care for toots, materials, and equipment 

3. Produce quality products 

4. Follow directions for a specific task 

5. Follow one-stop directions correctly 

6. Follow multiple step directions correctly (e.g., two to five steps) 

7. Show recognition of authority figures and supervisors and their roles 

8. Display loyalty to school/company/job 



N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 234 



WORK RELATED SOCIAL SKILLS (Competencies that will enable an indKndual to establish efTective interper&onal relationships on 
the job) 

1. Display behavior that is age appropriate N 1 2 3 4 

2. Work effecti'^ely under different styles of supervision N 1 2 3 4 

3. Work cooperatively as a member of a team N 1 2 3 4 

4. Get along and work effectively with people of different personalities N 1 2 3 4 

5. Show up regularly and on time for activities and appointments N 1 2 3 4 

6. See things from another point of view N 1 2 3 4 

7. Engage appropriately in social interaction and situations N 1 2 3 4 

8. Speak with others in a relaxed and self-confident manner N 1 2 3 4 

9. Compliment and provide constructive feedback to others at appropriate times N 1 2 3 4 

10. InKiate and maintain friendly conversations with another Individual N 1 2 3 4 

11. InKiate, maintain, and draw others into friendly group conversations N 1 2 3 4 

12. Join in friendly group conversations N 1 2 3 4 

13. Express complaints appropriately N 1 2 3 4 

1 4. Avoid arguments N 1 2 3 4 

15. Help others without being asked N 1 2 3 4 

16. Help others when asked N 1 2 3 4 

17. Show control of emotions and behaviors N 1 2 3 4 

18. Demonstrate appropriate manners in a social setting N 1 2 3 4 

SELF HELP/lNDEPENDEffT LIVING SKILLS (Competefwies that will enable an individual to live independently 
in the community) 



The student can: 

1. Provide correct personal information when asked (i.e., name, address, 
phone number, social security number) 

2. Read and display recognition of street signs 

3. Cross the street independently 

4. Use public transportation (e.g. bus, taxi) 

5. Demonstrate ability to order food from a menu 

6. Demonstrate the ability to read labels (e.g. food, packages) 

7. Purchase food and groceries 

8. Buy and care for personal clothing 

9. Select clothing appropriate for weather and activity 

10. Demonstrate ability to use laundry products appropriately 
(e.g., at home, commercia! facilities) 

1 1 . Perform basic household tasks 

1 2. Ingest medication as prescribed 

13. Correctly count money and make change 

14. Demonstrate appropriate money management through planning 
and budgeting 

15. Conduct bank transactions (e.g., checking account, savings account) 

16. Obtain help In emergency situations 

17. Plan use of leisure time for hobbies and recreational activities 

FIGURE 7-3 (cent) 

Source: Carper Portfolio, University of North Texai 
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N 1 2 3 4 

N 1 234 

N 1 2 3 4 

N 1 234 

N 1 234 

N 1 2 34 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 2 3 4 

N 1 234 

N 1 234 

N 1 2 34 

N 1 2 3 4 

N 1 2 34 

N 1 2 3 4 
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IL For Evaluating Work Behaviors and/or Habits 



Looking through the multitude of Rating Scales being used to assess 
behavior pertinent to a profession, you can see a common denominator. 
Even though the Rating Scales were independently constructed by different 
programs or used in different schools, a "Top Ten" listing of common 
behaviors are repeatedly targeted for assessment. 

Often called "soft skills", or even the "warm fuzzies", these qualities generally 
make a worker: 

• dependable, 

• efficient, and 

• a team member that can contribute positively to collective efforts 
resolving problems to "get the job done". 

The fact that the same qualities are so frequently listed will come as no 
surprise to anyone who has read the SCANS report, or other studies of 
what students need to develop to succeed in today's workplace. So in the 
examples we present here, you won*t find major differences in the actual 
listings or descriptions of the behaviors to be rated. 

The differences we seek to illustrate are in the ways that the habits can be 
rated. 



ra- Figure 7-4 (next page) shows a "Situational Assessment of Work 
Habits" that uses an open continuum to rate from one described 
extreme to another. 



^ Figure 7-5 (pages 13-14) shows a "Supervised Practical Experience 
Evaluation Form" in which a student's performance on the job can 
be rated according to three established levels of proficiency. 
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student 
Rater 



SnilATKDNAL ASSESSMEKT OF WORK HABITS 

Date 

School 



On the basis of your knowledge of the student, assess him/her on the following continuum: 



X 

APPUES 



MCLMED 
TOX 



MCUNH) 
TOY 



Y 

APPues 



1. MmATIVE 

Student usually follows 
through on instructions. 

2. QIMUTY0FV«3HK 

Student's work is done 
correctly and carefully. 

aOUAUTYOFWORC 

student's productivity is 
consistently high. 

< student rarely wastes time. 

5. HUMAN RELATXDNS 

Student relates well to peers. 

aStudent works well on joint 
tasks. 

7. ACCePTANCE OF AUTXDnmr 
Student accepts unpleasant 
tasks when assigned. 

iL AnmioE 

Student is neat, alert, and 
involved. 

SL Aaurr TO FOiJUQW MsnrnucTXDNs 
Student can understand most 
verbal directions. 

ia Student can understand 
most written directions. 



11. PQPBISTeJCC 

student stays on difficult tasks. 

12. TEACHBfS OVBWl 
REACTON TO SnJDEHT 

Student should succeed in 
vocational program of his/her 
interest. 



Student fails to follow 
instructions. 

Student's work is not 
done correctly or 
carefully. 

Student's productivity is 
low. 

Student often wastes 
time. 

Student does not relate 
well to his peers. 

Student does not work 
well on joint tasks. 

Complains about 
unpleasant tasks when 
assigned. 

Student is sloppy, 
sluggish, and 
uninvolved. 

Student cannot 
understand verbal 
directions. 

Student cannot 
understand written 
directions. 

Student does not stay 
on difficult tasks. 

Student will have 
difficulty succeeding in 
vocational program 
without assistance. 



Please recommend any support service that would benefit this student in realizing success in future vocational programs. 

ESL Support Curriculum Modification 

Remedial Math Counseling by School counselor 

Remedial Reading Other 

Job seeking skills 



Source: fVorth C«/oHn« VocatfonaJ Association 
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Supervised Practical Experience Evaluation Form 



Directions: Evaluate and rate the performance of the student listed below. 
Complete the form (Front and Back) using the Ratings (described below). 
Add comments about the student's performance and sign the form. 
Discuss the evaluation with the student. 

School/Center 

Training period 



RATINGS: Advanced - superior performance, demonstrates outstanding skill 
Profficiont- competent performance, demonstrates satisfactory skill 
Unsatisfactofy • Inadequate performance, skill needs much improvement 



AcSvanced Proficient Unsatisfactory N/A 
1. Personal Skills 

Appearance 

Attitude 

Time management 

Ethical behavior 




Student 
Job title 



Interpersonal Skills 
Cooperation 
Interaction 

Shared responsibilities 
Leadership 
Conflict resolution 



III. Thinking & Problenn Solving Skills 
Creative thinking 
Critical problem solving 
Information gathering 
Mathematical reasoning 

IV. Communication Skills 

Speaking 
Listening 

Following directions 
Writing 

Telephone skills 

V. Occupational Safety 

Understanding 
Materials and equipment 

VL Technology Literacy 

Basic technology 

Overall Rating 



Figure 7-5 

Sourco: Career-T^chntcaJ Attsossmont Pro|oct Portfolk) > p. 11 1 , Toacher Guidebook 
^ CiUifomia Dopartmont of Education and Far West Laboratory Page 1 3 
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Complete this section to evaluate the student's performance on additional Model Curriculum. Performance Standards, 
areas of training, and/or occupational competencies. 

Area Cluster 



Standards, Areas of Training, Advanced Proficient Unsatisfactory N/A 

Competencies 



3. _ 

4, _ 

7._ 
10. 



Comments on student's performance. 



Evaluator's signature Date 

Title 

Student's comments (optional) 



Student's signature Date 

instructor's signature Date 

(Attach training plan, if available. Store completed evaluation form in student's portfolio.) 

Figure 7-5 (cont) 

Source: Careor'Tochnlca] Asscssmont Project Portfolio - p. Ill » Teacher Notea for Work Samples 
CaiKomia Dopartmont o( Educolion and Far West Laboraiofy 
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Figure 7-6 (next page) shows "A Work Behavior Rating Scale" that 
rates observed students' performance in degrees of reaching 
"competitive standards in employment". 



«3r Rgure 7-7 (page 17) shows a "Worklink^ Teacher Rating" that 
makes a compact, computerized rating of a student's classroom 
attitude and effort. Educational Testing Service (ETS) developed 
the five-point scale going from "Highest" to "Lowest" as part of the 
results given when a student takes the "Worklink™" test battery. 
ETS designed the standardized tool for giving a work-related 
synopsis of high school performance. 



CroGspaths Managoment Systoms, Inc. 



Page 15 



Rating Key: 
E BccoUont 
VG Veiy Good 

G Good 
F Fair 



P 

NA 



Poof 

Not Applicable 



WORK BEHAVMDR Rating Scale 

Performance exceeds usual competitive standards in employment. 

Performance meets usual competitive standards in employment. 

Good performance, not quite equivalent to competitive standards in employment. 

Although generally acceptable, considerable improvement and/or consistency required to attain 

competitive standards. 

Below average performance, behavior requires special supervisory or professional staff attention. 



Woiicef Trait 


Rating 


Comments 


1. Attendance 






2. Punctuality 






3. Grooming, hygiene, dress 






4. Mood 






5. Understanding of oral instructions 






6. Understanding of written instructions 






7. Understanding of instructions when shown 






8. Overall memory for instructions 






9. Attention and concentration 






10. Work methods and organization 






11. Quality of work performed 






12. Speed and accuracy of tasks completed 






13. Consistency of tasks performed 






14. Safety awareness and habits 






15. Motivation for work 






16. Initiative 






17. Adaptability (i.e., acceptable reaction to 
change in work assignment) 






18. Frustration tolerance (i.e., ability to react 
appropriately to unpleasant or monotonous 
tasks 






19. Overall ability to get along with co-workers 






20. Ability to work on tasks with co-workofs 






21 . Acceptance of supervision and/or authority 






22. Tolerance for criticism from supervisor 






23. Amount of supervision required after initial 
instruction period 






24. Reaction to close supervision 






25. Ability to verbalize complaints/issues 
appropriately 






Sourc«: M«/1a Zakula, M A., C RC. 1968 


Figure 7-6 
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Student's Name: 



Worklink™ Teacher Rating 



Course Title: 



School Name: 



Teacher's Signature 



HIGHEST RATING = A LOWEST RATING = E 



1. Attendance: 

A. Virtually perfect attendance D. Many absences 

B. Seldom absent E. Seldom attended class 1. A B C D E 

C. Some absences 



Z Punctuality 

A. Virtually always on time D. Often late 

B. Seldom late E. Seldom on time 2. A B C D E 

C. Sometimes late 



3. Completion of Assigned Work (Whether on time or not) 

A. Always complete D. Often incomplete 

B. Most often complete E. Most often incomplete 3. A B C D E 

C. Often complete 

4. Punctuafity in Handing in Assignments (Whether complete or not) 

A. Always on time D. Seldom on time 

B. Most often on time E. Never on time 4. A B C D E 

C. Often on time 



5. Paying Attention in Class 

A Always paid attention D. Occasionally paid attention 

B. Most often paid attention E. Scarcely ever paid attention 5. A B C D E 

C. Often paid attention 

6. Effectiveness As A Group Member 

(Please omit if you had insufficient opportunity to obserye performance in a group) 

A. Always effective D. Seldom effective 

B. Generally effective E. Never effective 6. A B C D E 

C. Sometimes effective 



7. Cooperation in Class 

A. Always cooperative D. Occasionally cooperative 

B. Most often cooperative E. Scarcely ever cooperative 7. A B C D E 

C. Often cooperative 



FIGURE 7-7 



Source: Worklink^x . developed by ETS, 1992. Page 17 
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WHY? 



Illustrating the WHAT of Rating Scales should also serve to show WHY 
they can be such an effective and far-reaching assessment strategy, 
especially for bridging language barriers. 



Perhaps what is less well known or understood is why it is so important 
to have the student also be the one doing the ratings. 



Businesses are stressing the need for workers to have "self-management 
skills." Educators and researchers are pinpointing "metacognition" as the 
foundation needed to assimilate knowledge into meaningful growth. 
Both the business world and the schools seek the same thing: having 
students able to monitor their own learning process. 



To really be able to go fon/vard with learning, students must develop 
internal "rating" systems. They have to find ways that work for projecting 
into a problem and getting back information that will tell them what 
needs to be done next. 



Rating Scales can give students a format to follow for developing those 
skills, and the practice needed for learning them. By using Rating Scales 
to rate themselves, students see: 

• What they are expected to know 

• What they already know 

• How well they know it 
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• How effectively they are using what they know 

• How much they still have to learn 

• How to best go about doing what still has to be done 



Obviously, these are strong reasons why Rating Scales can be an 
effective assessment/instructional strategy that brings the student actively 
into the process. 



However, like every technique or tool. Rating Scales also have limitations 
which must be recognized and over-interpretation errors to be avoided. 



On page 2 of this chapter we list the disadvantages that can skew 
the results. 



All three of the drawbacks stem from the same source: the differences in 
people and between people that can impact the answers in an 
uncontrollable manner. 



Obviously, personality differences cannot be submitted to statistical 
control. But they can be better understood, especially in terms of how 
they can influence people's responses to questions about their own 
abilities and feelings. Such understanding can help turn "weaknesses" 
into strengths. 



For Example: 



1) A person's self-esteem. It would make matters less complicated 
if people who think highly of themselves always record high ratings for 
themselves, and visa versa for those who have a poor self-image. 
Unfortunately, such neat lines cannot be drawn. 
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Students can appreciate their own capabilities, but give themselves low 
marks at times because they see they haven't accomplished yet all that 
they plan to achieve. 



Students can seriously question if they will ever do anything right, yet try 
to make a brave front for fooling others and themselves by putting high 
marks on their ratings. 



These are valuable insights to get about a person you have to guide, 
and you can find that out about a student by judicial use of Rating 
Scales. 



2) Social/cultural norms for expressing and developing self- 
esteem. A person's self-esteem is often reflected through the prisms of 
gender and culture. 

This was seen even before recent awareness stud^^s started calling for 
reassessments of strategies. As early as 1969, statistics revealed that 
American female students underestimated future performance on 
intelligence and academic tasks while males tended to overestimate 
themselves on the same tasks. If a student is from a culture in which 
females are strongly expected to play passive, secondary roles, in which 
they downplay and doubt signs of self-assertions, the difference would 
be even more noticeable. 



There are also many different cultures which stress humility as a virtue. 
Students raised to value humble self-appraisement would think it rude to 
rate themselves highly. This could be the possible explanation for the 
statistical results gathered in a study that field-tested a new way to assess 
the English comprehension of non-native speakers. After completing the 
English ability test, students filled in self-Rating Scales. An analysis of the 
results showed that Japanese, Chinese and other Asian students tended, in 
comparison with other students, to rate themselves lower In ability than their 
scores actually registered. (Manning, 1987). 
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HOW? 



Often the hardest question with assessment strategies is how do you 
take what sounds like a good idea and actually put it into practice? To 
implement Rating Scales as successfully and painlessly as possible, 
some answers to the following questions will be helpful: 



0* 



HOW are Rating Scales constructed? 



HOW can Rating Scales be modified to better assess LEP 
students? 



^ HOW can Rating Scales be used by students to assess 
vocational situations and their own skills? 
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HOW To Construct Rating Scales 



We introduced this chapter by explaining that a Rating Scale is one of 
the simplest forms of assessment througn observation; that the format is 
essentially a listing of what you want to know; and that it judges the 
degree to which certain performances are effective. 

Therefore, to make a Rating Scale, there are two main coordinates to be 
determined and put together: 

1) A listing of what you want observed 

2) How you will "grade" what you observe 



L Making A List of What You Want To Assess 



The steps for doing this are as clear and easy as using the final product 
should be. 



1) First, you think of all the activities involved in doing the task or 
behavior that you want the student to accomplish. 

2) Put the key elements, or steps of the task, in the sequence in 
which they occur. This sequenced listing of steps collectively 
gives you the process a worker goes through to complete that 
task. 

3) With some tasks, there is also a desired product ( eg., a mailable 
letter, a workable piece of machinery that has been repaired or 
replaced). If that is the case, characteristics (appearance, 
condition, etc.) of the final product can provide useful criteria for 
assessing performance of the set task. 
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The appropriate characteristics of the desired product would then 
be added to the list of steps involved in producing the product. 

4) At this point, you will have a complete list of all the student must 
do to perform the task correctly. Now you need an objective 
method to evaluate the student's performance, the second 
coordinate by which you will situate your observations. 





II. Determining How the Parts of the Perfonnance Will Be Rated 



When a person performs an assigned task, whether in school or on the 
job, the results are judged by three bottom line criteria: 

1) Effective 

2) Sometimes Effective 

3) Needs Work 



From a teacher's perspective, the evaluation will also have the dimension 
to determine if a student has learned enough to function: 

1) Independently 

2) With Assistance 

3) Does Not Perform 



The possibilities for how the basic evaluation points can be expanded, 
reduced or altered are obviously varied, as we have shown through 
representative samples of Rating Scales being used in the field. A 
sun/ey of the different types being used revealed that there are 5 formats 
which are the most frequently developed. 
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Rgure 7-8 (next page) gives an explanation and an example of the 
five most frequently used Rating Scale Formats. 



Careful consideration should be given to choosing the format for the 
Rating Scale that you design. As one experienced VESL instructor told 
us, "Each Rating Scale frames the responses. They give the guidelines 
for how you are going to think about and answer the questions." (Answer 
on evaluation survey that was part of the Field Test at El Camino 
Community College, Torrance, California). 



Cultural/linguistic factors are already influencing the possible responses 
of LEP students, even before they begin to follow a format. The special 
considerations that need to be understood and addressed in developing 
the most open possibilities for LEP learners are specified in the section 
that follows. 
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Rating Scale Formats 


TYPE OF SCALJE DESCRIPTION RESPONSE FORMAT 

ITEM: "1 UKE SCHOOL" 


Checklist 


One of a variety of ways 
to simply check an item 
as appropriate. 


Check one: 

Yes 
No 


Category rating 


Provides words that allow 
the respondent to select 
an answer that best 
characterizes the 
response to the item. 


Select the answer {or item 
aajective, etc.; tnat oest 
completes the statement. 


Numerical rating 


Provides words and 
numerical representations 
inai suggesi equai 
inten/als on a continuum. 
Can suggest 
comparability between 
items. 


Circle one number: 
1. No agreement 
^. LOW ayrecriieriL 

3. Medium agreement 

4. High agreement 

5. Total agreement 


Graphic rating 


A linear representation 
that suggests a 
continuum with equal 
inten/als. One of the best 
procedures, combining 
words with space. 


Check on the scale: 

J \ \ \ L 

Very Some Some Not Very 
much what much little 


Forced-choice item 


Requires the selection of 
a response from among a 
set of equally attractive, 
unpleasant, or otherwise 
similar choices. 


Check one: 

Never 

Almost never 

Sometimes 

Almost always 

Always 



Figure 7-8 

Source: Guerin & Maier. 1983. p. 101 



Page 25 



170 



Rating Scales - Chapter 7 




HOW Rating Scales Can Be Modified To Better Assess 
LEP Students 



The idea of Ratings Scales, per se, can stand on its own as one of the 
fairest assessment concepts for use with LEP students. The chance to 
actively show a skill instead of struggling to explain it presents 
possibilities that can enable, rather than limit, the learning process. 



The presentation and application of Rating Scales determine if the 
concept will reach its potential. Like every assessment strategy, a Rating 
Scale has some drawbacks and possible pitfalls. 

^ See " Advantages-Disadvantages" in this Chapter, page 2. 



If those are kept in mind as the general guidelines, and the following 
specific points are also added, then you will incorporate the open 
flexibility that offers and elicits responses from different points of view. 



1) Allowance for Rating Scale to Show Progress Made in English. 
Learning a language is a valuable skill, one that LEP students are 
acquiring at the same time they are developing vocational 'tools of 
the trade." If the area of vocational training is one where mastery of 
English is essential, devise a listing of actual work skills and allow 
the rating to reflect progress in being able to do each skill in 
English. 



»y Figure 7-9 (next page) is an example of a Rating Scale that rates the 
competencies taught in a Data Entry training program. The training 
program has a strong VESL component, and this is captured by 
assessing technical skills in relation to progress made in being able 
to perform them in English. 
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DATA EHXKf VESL COMPETENaES 

Bating Scato: 

4 - Skilled - Uses English appropriately without prorrspting or assistance: is easily understood; almost always uses correct 
grammar, spelling, and pronunciation. 

3 - Moderately skilled - Uses English appropriately, but may require occasional prompting or assistance; usually easy to 
understand - good control of grammar, spelling, and pronunciation. 

2 - Acceptable skill - Uses English in a generally appropriate manner; requires some prompting or assistance; can be 
understood, but may take some effort - mistakes in grammar, spelling, and pronunciation do not usually inhibit ability to be 
understood; mistakes may be numerous. 

1 - Limited Skill - Has drfficutty using English appropriately; often requires prompting or assistance; often difficult to understand - 
does not control grammar, spelling, or pronunciation; mistakes inhibit ability to be understood. 

0 - No skill/Experience in this area ■ Cannot use English appropriately; does not respond to prompting or assistance; extremely 
difficult or impossible to understand - pronunciation may be almost incomprehensible. Use this rating when student has not 
attended corresponding classes and has not made up work. 



CCMPETENOES TO BE RATED 


4 


3 


2 


1 


0 


ORIENTATION 


1. Introduce oneself/state background. 












2. Ask information question. 












3. Ask for clarification. 












TELEPHONING 


t 

1. Answer phone in office setting. 












2. Give and take a phone message. 












3. Transfer a call. 












4. Report a message. 












5. Telephone manners. 












6. Reacting to information. 












7. Terminating the call. 












8. Call for information about a job opening. 












9. Call to request a job application. 












10. Call to make an appointment for an interview. 












11. Ask for clarification of/confirm information in telephone calls. 












12. Follow up an Interview by phone. 












13. Understand recorded telephone message on local job hot lines. 












ALPHABETTZING/nUNG/ABBREVlATIONS/SPELUNG 


1 . Alphabetize vocabulary words. 












2. Learn state, territory, street names. 












3. Learn to spell American names. 












4. Learn to spell terminology/words. 












5. Index names. 












SOCIALIZING ON THE JOB 




1. Greet a supervisor, co-worker. 












2. Roleplay a social conversation in English with a supervisor/co-worker. 












3. Compare jobs and countries. 













Figure 7-9 



Source: Lopez-Valadez & Reed, 1989 
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2) Take into account accutturation factors. If you are asking students 
to rate interests or experiences, allow options that give allowances 
for lack of exposure or experience with activities common in 
America, but maybe not yet familiar to LEP students. 



^ Figure 7-10 (next page) shows how a good practice can be made 
even better with only a slight, but important modification. 



A teacher at Bell Multicultural High School, a culturally diverse 
school in Washington, D.C., starts the school year by having her 
students rate their preference for various activities. She uses the 
results to have projects included in her lesson plans that would be 
relevant and appealing to the different students. 



The format is simple and easily translatable for LEP learners, in 
deciding if they like, dislike, or are neutral about possible learning 
situations. 



Initially, however, an important area was not included. By adding 
the option of "Never Done \V\ you can get essential experiential 
information that can determine the amount of preparation needed 
before approaching certain activities. In the case of LEP immigrants, 
this also gives the teacher important information about the students' 
level of acculturation. 
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Date: Group: 
First Name: 
First Language: 



Activity Preference inventory 

Teacher: 
Last Name: 
Second Language: 



Grade Completed: 
Age: Gender: 
Nationality: 



* Fourth column was not included in the original fonn. It was added as an example of modificatioiL 

Please respond to the following questions to iielp us improve your education program. Mark the column that indicates your 
preference: 



ACTMTY 


ICkeit 


It is OK 


1 don^ like H 1 


Never done rt* 


Watching the TV news 










Working alone 










Watching videos 










PlA\/infl aArriA^ 










Wrrtinn Iptter^ 










Working in groups 










Reading the newspaper or magazines 










Working with numbers 










Discussing issues 










Listening to lectures 










Working in pairs 










Reading books 










Reading maps and charts 










Writing descriptions 










Reading stories 










Conversing with others 











Writing stories 










Drawing 










Interviewing people 










Using a TV camera 










Using a tape recorder 










Listening to authentic conversations 










Doing grammar and pronunciation 










Listening to music 











Source: M«. Ana Vasquez, Bell Mutticultural High School, Washington, D.C. 
Modification Addition Suggostod by. Dr. Ruth Potkofl. Consultant 
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3) Use a format that limits language and opens up the most options. 

83r Figure 7-8, on page 25 illustrates the five most common formats 
used when creating a Rating Scale, 

In terms of the possibilities offered by each, a progressive analysis 
shows which ones hold the most promise as procedures to help LEP 
students more fully and fairly respond. 

• Checklists (reducing options to usually 'Ves" or "no") are 
too narrow a scope for any in-depth evaluation processes, 

• Category Ratings (providing words to best complete or 
characterize responses) can become bogged down in 
linguistic complexities and shadings. 

• Forced Choice items are exactly that-- forcing you to 
choose between a few given options that at times can offer 
no real personal preference or option. Furthermore, studies 
have confirmed that a person's choices can vary according 
to different settings and context, making this format 
perhaps the most vulnerable to cultural biases and 
inaccurades- 



• That leaves the two formats of 

• Graphic Ratings (combining few words with space 
on a continuum) or 

• Numerical Ratings (giving number representations to 
open ranges of choices) 

as the most promising possibilities for LEP learners to 
actively participate in assessment as a development 
process. 
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HOW Rating Scales Can Be Used By Students To 
Assess Vocational Situations and Their Own Skills 



The importance of involving the student actively in the assessment 
process has been stressed repeatedly throughout this Handbook. 

^ Chapter 4 - Assessment, pages 24-25 

^ Chapter 5 - Interviews, pages 4; 16-27 

^ Chapter 6 - Cloze, pages 21-22 

83P Pages 1 8-20 of this chapter 

Rating Scales offer a way to bring students actively into an evaluation of 
their learning. For LEP students, Ratings Scales also add the element of 
independence to their involvement. This is done by having the students 
do the rating as well as the instructor. 

Student self-ratings give the rare freedom for LEP students to by-pass 
on-going oral translations and develop their thoughts and criteria within 
their own language structures. Granted, the given format might need an 
initial linguistic clarification, but then students will have the basis for doing 
repeated assessment exercises on their own. 

The examples that follow of how this can be done illustrate three different 
aspects that a student can be asked to analyze through a Rating Scale. 



Figure 7-11 (pages 32-34): 



"Comprehensive Analysis of A 
Vocational Training Program" 



Figure 7-12 (pages 35-36): 



"SCANS Self Appraisal' 



ly Figure 7-13 (page 37): 



"Performance Checklist: Welding 
Vehicle Body Parts" 
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COMPREHENSIVE ANALYSIS OF 
A VOCATIONAL TRAINING PROGRAM 

Vocational Program Trtlo: 

Location: 

Teacher 

L OccupationaJ Cluster 

Match the program to an Occupational Cluster. Circle the cluster which best describes the job. 

1. Professional, Technical and Management Occupations 

2. Clerical and Sales Occupations 

3. Service Occupations 

4. Agricultural, Fishery, Forestry, and Related Occupations 

5. Processing Occupations 

6. Machine Trades Occupations 

7. Bench Work Occupations 

8. Structural Work Occupations 

9. Miscellaneous Occupations 

II. Unploasant Wodcing Conditions 



Rate each working condition as applied to the vocational/training program. Mark an 'X' in the space which best 
describes the working conditions. 





Continuous 
Exposure 76-100% 


Extensive 
Exposure 51-75% 


Intermittent 
Exposure 16-50% 


Rare Exposure 
1-15% 


No Exposure 
0% 


1. Hazardous 












2. Loud Noises 












3. Vibrations 












4. Intense Ught 












5. Hi/Lo Temp. 












6. Fumes 












7. Odors 












8. Toxic Materials 












9. Dust or Lint 












10. Poor 
Ventilation 













NOTE: The percent refers to the amount of time each program is involved in a particular condition or skill. For example if the 
program Is not hazardous at any time, it Is marked in the No Exposure 0%; while a program that exposes the worker to 
loud noises a little more than half the time is marked in the space Extensive Exposure 51-75% 



FIGURE 7-11 

Source: Kapos & Parrish (1987) 
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111. Job Charadenstics 

Mark an 'X* in the space which best describes the characteristics of the vocational training program. 





Continuous 
76-100% 


Extensive 
51-75% 


Intermittent 
16-50% 


Minimal 
1-15% 


None 
0% 


DATA 


1 . Money 
Transactions 












2. Measurements 
and Gages 












3. Weights and 
Scales 












4. Calculations 












5. Applied 

Reading Skills 












PEOPl£ 


6. Working 
with Others 












7. Supervised by 
Employer 












8. Co-worker 
Teamwork 












9. Customer 
Interaction 












10. Directing 
Work of Others 














11. Production 
Stress 












12. Precision 
Quality Control 












13. Short Term 
Task Repetition 












14. Setf-Control of 
Work Place 












15. Cleanliness 













Source: Kapes & Parrish (1987) 



FIGURE 7-11 (con'l) 
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IV. Sensofy Capacity 



Mark an "X' In the space which best describes the sensory capacities needed to perform the vocational training program. 





Continuous 
76-100% 


Extensive 
51-75% 


Intermittent 
16-50% 


Minimal 
1-15% 


None 
0% 


1 . Visual 
Discrimination 












2. Color Vision 












3. Spatial and 
Form Perception 












4. Tactile (touch) 
Sensitivity 












5. Hearing and 
Talking 













V. Physlcai Capacity 

Mark an ')(* in the space which best describes the physical demands of the vocational training program. 





Continuous 
76-100% 


Extensive 
51-75% 


Intermittent 
16-50% 


Minimal 
1-15% 


None 
0% 


1 . Use of Both 
Hands 












2. Hand Stre gth 












3. Eye-Hand 
Coordination 












4. Finger 
Dexterity 












5. Prolonged 
Sitting 












6. Standing and 
Walking 












7. Lifting and 
Carrying 












8. Balancing and 
Climbing 












9. Total Body 
Coordination 












1 0. Physical 
Endurance 













VI. Educatiooal Requirenionts 

Circle the educational requirements which apply 

0. Reading and math skills not required. 

1. Third grade reading and math skills. 

2. On-the-job training wKhout completing high school. 

3. High school diploma or G.E.D.. 

4. Vocational technical training. 

5. College degree. 

FIGURE 7-11 

Source Kapes and Parrish (1987) 
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SCANS SELF-APPRAISAL 



This is my personal opinion of how well 1 have mastered each of the SCANS skills/ competencies. 
1. BASIC SKILLS : Reading, writing, arithmetic and mathematics, speaking and listening 

Good Skills OK Need Help 



2. THINKING SKILLS : Thinking creatively, making decisions, solving problems, seeing things in the 
mind's eye, knowing how to learn, and reasoning 

Good Skills OK Need Help 



3. PERSONAL QUALITIES : Individual responsibility, self-esteem, sociability, self-management, 
and integrity 

Good Skills OK Need Help 



4. RESOURCES : Allocating time, money, materials, space and staff. 

Good Skills OK ^ Need Help 



5. INTERPERSONAL SKILLS : Working on teams, teaching others, serving customers, leading, 
negotiating, and working well with people from culturally diverse backgrounds 

Good Skills OK Need Help 



6. INFORMATION : Acquiring and evaluating data, organizing and maintaining files, interpreting and 
communicating, and using computers to process information 

Good Skills OK Need Help 



7. SYSTEMS : Understanding social, organizational, and technological systems, ntonitoring and 
correcting performance, and designing or improving systems 

Good Skills OK Need Help 



8. TECHNOLOGY : Selecting equipment and tools, applying technology to specific tasks, and 
maintaining and troubleshooting equipment 

Good Skills OK Need Help 



Figure 7-12 

Sovrco: Los Angolos Unlfiod School District (1992) Page 35 
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PLAN FOR SELF-IMPROVEMENT 



In order to become a more productive citizen and worker, it is important that I improve in the following 
SCANS skill: 



Although there are many ways to improve this particular skill, I have selected the following three 
activities to help me: 



1. 



Start Date: Planned Completion: Actual Completion: 



2. 



Start Date: Planned Completion: Actual Comipletion: 



Start Date: Planned Completion: Actual Completion: 



Signed: 



Figure 7-12 (con't) 

Source: Lo3 Angolos Unified School District 
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PEWORMAhK^e CHECKLIST 
Welding Vehicle Body Parts 



Student 
Evaluation 



InstAictor 
Evaluation 



YES 



NO 



YES 



NO 



1 . Read work order. 



2. Seiected toois. 



3. Selected proper welding material. 



4. Aligned replacemant panel property 



5. Spot welded to joining panel. 



6. Filed and finished joint. 



7. Matched original contours of body. 



8. Replaced tools. 



9. Cleaned area. 



FIGURE 7-13 

Source: Bradley and Friedenberg, 1988 
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WHERE? 




Where Examples of Rating Scales Have Been Developed 



Career Portfolio 

University of Texas at Austin 

Extension Instruction and 

Materials Center 
Post Office Box 7218 
Austin, Texas 78713 
Stock No. SN 005 I 



Scans Self-Appraisal 
Los Angeles Unified School 
District 

Superintendent's Office Special 

Projects 
1208 Magnolia Avenue 
Gardenia. California 90247 
Contact: Dr. Gabriel Cortina 
Assistant Superintendent 




Sources of Rating Scales Used In This Chapter 



Bradley, C, and Friedenberg, J., Teaching Vocational Education to 
Limited English Proficient Students . (1988). Bloomington, Illinois: 
Meridian Education Corporation. 

California Department of Education and Far West Laboratory, Career- 
Technical Assessment Project Portfolio: Teacher Guidebook. (1992). San 
Francisco, California: Author. 
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Educational Testing Sen/ice, Worklink^: Teacher Rating. (1392). 
Princeton, New Jersey: Author. 

Guerin, G. and Maier, A., Infornnal Assessment in Education^ (1983). Palo 
Alto, California: Mayfield Publishing Company. 

Kapes, J. and Parrish, L, and Vansickle, T,, Vocational Assessment 
Handbook, (1987). College Station, Texas: Texas A and M University, 

Lopez-Valadez, J. and Reed, T,, Building Competencies to Serve LEP 
Vocational Students: Inservice Manual. (1989). DesPlaines, Illinois: 
Northwest Educational Cooperative. 

Los Angeles Unified School District, Portfolio Process. (1992). Los Angeles, 
California: Author. 



Thomas, R., Grover., J., Cichon, D., Bird, L, and Hams, C, Job Related 
Language Training for LEP Employees: A Handbook For Program 
Developers . (1991). Washington D.C.: Development Assistance 
Corporation. 

Thomas, S. and Coleman, N., Vocational Assessment Training Manual . 
(1988). Raleigh. North Carolina: N.C. Department of Public Instruction, 
Division of Vocational Education. 



University of North Texas, Career Portfolio, (1991). Denton, Texas: 
Author. 



Zakula, M., "Work Behavior Rating Scales*'. Paper presented at The Sixth 
National Forum on Issues in Vocational Assessment, March 4-6, 1993. 
Virginia Beach, Virginia. 




Study Cited on Student Self Ratings 



Manning. W., "Development of Cloze-Elide Tests of English as a Second 
Language", in Test of English as a Foreign Language (TOEFU: Research 
Reports . Educational Testing Sen/ice, Report 23, April, 1987. 
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PORTFOLIOS 




WHAT? 



Portfolios as a teaching/assessment tool has emerged as almost a 
sacrosanct buzzword within the new wave of assessment approaches, yet 
many craftsmen in vocational education fields have been producing 
versions of them for years. A good cabinet maker, for example, knows that 
people want to see samples of her work before hiring her to do a job. To 
show what she can do, she will have some of her finished designs available 
for public viewing. 



Basically, that is what a portfolio is all about: 



An assembly of products 



which demonstrate a 
learned process 



that has led to 



• 




Crosspalhs Managomont Systoma, Inc. 
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Some people prefer to call it a "processfolio", to emphasize the importance 
of showing growth and development through a collection of work. 

Others say that the importance is in having a portable showcase of 
completed products you can carry around with you, a binder or folder filled 
with your accomplishments that demonstrate your competence to those 
who need to see it. 

Whether you are in the "process" camp or the "product" camp, a portfolio 
has an important distinguishing feature, which makes it different from just 
being a folder or collection of work: 



A portfolio involves a selection process; decisions are made 
about what pieces are included and which ones are rejected. 



The student and the teacher are constantly evaluating work that is done as 
a possible contribution to the ultimate goal of accomplishment. You choose 
the best; teachers and students become allies to show what students can 
do rather than what they c^n't. 
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WHY? 



Using portfolios as an instruction/assessment strategy has created a 
bandwagon movement with a strength that is changing more than the way 
we teach and test. Portfolios have worked their way into laws, not just 
classrooms, in states such as Vermont and Michigan. Perhaps more than 
any other form of "alternative" or "authentic" assessment approaches, 
portfolios have become a topic of debate with devoted advocates, and are 
being tried from kindergarten through graduate school. 

Why has the concept become so widespread and popular? Why do so 
many educators advocate portfolios as THE most promising practice in 
producing a real learning experience? 

Precisely the answer comes from there being no one answer. Portfolios are 
seen as filling a long list of needs and purposes. The California Student 
Assessment System has focused heavily on portfolios as part of the drive 
to revamp assessment to reflect new goals for curriculum and instruction. 
Teachers working with portfolios across the state compiled a list of 35 
different reasons why they felt that portfolios are so effective. 

Mitchell (1992) categorized the 35 purposes into 4 main areas, noting that 
many reasons easily intertwined in different areas. For this Handbook, we 
did a modified re-arrangement as a clear way of seeing why educators are 
developing and using portfolios in their classrooms: 

• As a Teaching Tool 



• For Assessment Purposes 



• For Professional Development 



Cf<>5»path8 Manogomont Syctonw, Inc. Page 3 
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1 . As A Teaching Tool 



Provides students with a sense of ownership, motivation, 
accomplishment, and participation. 

involves students in a process of self-evaluation. 

Helps students and teachers set goals. 

Builds in time for reflection about students' accomplishments. 

Individualizes instruction. 

Provides more reasons/opportunities to review a work. 

Sets up an apprenticeship situation. 

Connects reading, writing, thinking, and doing. 

Serves as a vehicle for publication. 

Aids in parent conferences. 

Gives importance to daily learning activities. 

Extends the amount of time devoted to practice in developing 
a skill. 



Z For Assessment Purposes 



Serves as an alternative to standardized testing. 
Serves as a job/college application vehicle. 
Demonstrates competency. 

Serves as a grade or end-of-year culminating activity. 

Provides program evaluation. 

Supplements state or district mandated assessment. 
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♦ Serves as a vehicle for changing the schools' conversation 
with the public. 

♦ Observes growth in second-language students. 

♦ Documents progress over time. 

♦ Validates how students learn a new language. 

♦ Looks at revision/process. 




♦ Assesses curriculum needs. 

♦ Studies curriculum and effective teaching practices. 

♦ Allows for better staff communication. 

♦ Reduces the paper load. 

♦ Identifies school strengths and needs for improvement. 

♦ Examines learning in different disciplines. 

♦ Builds a sequence in instruction. 

♦ Establishes an esprit de corps within departments and 
faculties. 

♦ Accounts for curriculum implementation. 

♦ Fosters professionalism and collaboration. 

♦ Evaluates the kinds of assignments given students. 

♦ Accommodates schoolwide projects: artwork, math graphs, 
writing from community, parents, teachers, custodial staff, and 
others. 

Cfosspethfl Managomont Systems, Inc. Page 5 
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HOW? 



Most teachers have heard the praises of portfolios and many would like to 
give them a try. But portfolios are not a spur of the moment inspiration or 
something that can suddenly be stuck into a lesson plan. They require 
extensive commitment and planning that can make many teachers feel 
overwhelmed, wondering how to start a portfolio with students, much less 
keep it going. 

When it comes to implementing portfolios with students who have difficulties 
with the English language, teachers can envision even more challenges. 

Portfolios are inclusive and therefore can initially seem very complex. 
However, they can also simplify many frustrations when it comes to 
demonstrating competency, especially for the students who are offered 
outlets of expression other than language. 

In this section, we outline the basic steps for seeing: 



0* 



HOW 



portfolios can open the assessment process for 
LEP students 



HOW 



to design a portfolio to fit you*' needs 



0* 



HOW 



to decide what demonstration data is needed 



0* 



HOW 



to collect the necessary data 



HOW 



to use the results 



Page 6 
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HOW Portfolios Can Open The Assessment Process 
For LEP Students 



The heavy reliance on writing is often an area of concern in using portfolios, 
especially with LEP students. However, one of the big advantages of 
portfolios is that a wide variety of strategies for expressive work can be 
called into play, at the same time that writing skills are being developed. 
A portfolio is a collection of ways of demonstrating a developed range of 
skills, rather than a focus on the present limitations of a particular language 
skill. 

Having so many possible strategies makes poitfolios not only the preferred 
assessment instrument, but also one of the fairest for LEP students 
according to many educators (Navarrette, 1990), 



Specific Portfolio Aspects That Highlight 
the Positive For LEP Students 



1) When using portfolios, teachers must emphasize that the students 
are not just preparing work for only a teacher to read and grade, but 
rather to appeal to a broader audience. This is of vital importance 
to LEP students vMo will be better prepared for their first interview 
with a concrete sample of their ability and potential in hand. Making 
LEP students aware that their work may be seen by future 
employers, community organizations, or school committees gives the 
extra motivation often needed for students with weak writing skills to 
make repeated editing attempts. 



2) Portfolios often involve teamwork efforts, and should definitely do so 
with LEP students. Through a team, approach for example, a student 
does not have to struggle on his/her own to find out how a concept 
can be better expressed; correct English can evolve out of a con- 
sensus approach. 
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3) Portfolios allow for, even require in many cases, mediums other than 
the written word to measure progress. Photographs, graphics, 
designs and constructed models are some of the many ways limited 
English proficient students can clearly illustrate their 
accomplishments and vocational competencies. 



Developing and Using A Portfolio Involves Following 
5 Basic Steps: 



I. 


Designing the portfolio 


II. 


Selecting the necessary data 


III. 


Collecting the necessary data 


IV. 


Analyzing the portfolio contents 


V. 


Using the results 



We will explain how each step can be done and give examples of how 
different public high schools are doing them. The examples are merely 
illustrations of specific points. They represent possible approaches to a 
part of the overall, organized strategy that is needed. 



EDITORS' NOTE: The reader should refer to the WHERE section 
of this Chapter, pages 43-49, to find the complete source needed 
to put the examples we use into their proper context. It is also 
important that the reader note that the examples given are excerpts 
from portfolio projects that have been copyrighted by their authors. 
Permission has been given for parts of portfolios to be reproduced 
here solely for the purpose of illustration in this Handbook. 
Any other reproductions must have permission directly from the 
original authors. 
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HOW to Design A Portfolio to Fit Your Needs 



The design of your portfolio depends on what you want it to demonstrate. 
Therefore, to evolve a design that will fulfill your needs, follow the 
sequenced process of: 



1) Determine specific learning goals on which the work will focus. The 
material in a portfolio is most useful when it reflects progress 
towards achieving clear and particular goals. The focus can be on 
content areas, on certain skills, or on program objectives, for 
example. The portfolio demonstrates what a person has learned. 
What do you want your students to learn? 



2) Identify perfomiance tasks and appropriate instruments. Once you 
have identified the learning goals and performance objectives to be 
demonstrated, decide how achieving those aims can best be shown. 
Which products provide clear proof of each determined ability? 



It is generally considered most effective if "mixtures" are generated 
by this selection process. 

• Include traditional and "authentic" assessment tools to give 
multiple indicators of a student's ability levels. Combining 
standardized tests' results with teachers' and students' 
designed results provides broad information. It gives a more 
balanced picture, especially in the case of LEP students who, 
due to policy mandates, have taken standardized tests that 
likely have been inappropriate and inaccurate reflections of 
abilities. It also gives mechanisms that allow for reviewers of 
the portfolio to cross-check the information that is presented. 
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• Establish "Required** and "Optional" items as points of 
evaluation. 

Required items give an element of consistency that can 
easily be communicated to other teachers or administrators. 

Optional items give the chance for on-going ratings of 
work in progress or of a student's particular interests and 
strengths. 

By including both types, teachers get the information to 
make instructional decisions, and students are encoi-.raged 
to participate actively in the portfolio's design and use. 



«3P Figures 8-1 through 8-6 are reduced copies of the Michigan 
Employability Skills Portfolio, a school system's basic portfolio format 
designed around the answers that came out of these 2 preliminary 
steps. The Michigan State Board of Education established 
partnerships with state businesses to form the Employability Skills 
Task Force. It was a collaborative effort to ensure that Michigan's 
high school graduates have the skills they need to be employable. 
Michigan employers indicated what skill areas are important on the 
job, and the Michigan Department of Education identified the 
learning instruments and performance tasks that fit the defined 
focus. 

The Michigan Employability Skills Portfolio is formatted into office-size 
folders for holding "evidence" of competency. There is a color coded folder 
for each of the 3 designated skill areas: 

^ Rgures 8-1 (next page) and 8-2 (page 12): Academic Skills 

^ Figures 8-3 (page 13) and 8-4 (page 14): Teamwork Skills 

»y Figures 8-5 (page 15) and 8-6 (page 16): Personal Management 

Skills 

A cross-referenced numbering system is used so that each folder will check 
the skill that is demonstrated with the work examples included in the 
folders. 
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Academic Skills help prepare you for future training, college and work. They include communicating, using 
mathematics, using science and technology, and solving problems. 



Communicatioii Mathematics 



Al. Read and 
understand \Mltten 
materials* 












A4. Understand 
charts and graphs* 












A2. Write in the 
language in which 
business is conducted* 












A5. Understand 
basic mathematics* 












A3. Speak in the 
language in which 
business is conducted* 












A6. Use 
mathematics to 
solve problems* 
























































































1 





















Science and Technology Problem Solving 



A7. Us© scientific method 
to solve problems* 






1 






A10. Use research and 
library skills* 












A8. Understand basic 
science* 












A1 1 . Use specialized 
knowledge and skills or 
know how to access it* 












A9. Use equipment and 
tools* 












A12. Understand 
systems and complex 
relationships* 













































































































♦These employabUity skills are necessary to help prepare you to enter the work force or obtain more 
education. 

FIGURE 8-1 

Source: Michigan Employability Skills Portfolio: Copyright © 1993, Michigan State Board of Education 
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EXAMPLES OF THINGS TO PUT IN MY PORTFOLIO 
Academic Skills 



SCHOOL RECOHDS 


A1. 


A2. 


A3. 


M. 


A5. 


A6. 


A7. 


A8. 


A9. 


A10. 


A11. 


A12. 


Transcripts w/class 

uooCi ipuoi 19 anu yiauvo Tor 

related classes 


























1 icIlloCitpi* lOi 

outside of school, e.g., 
vocationai cntr., college 


























r^VI llvVvlIlel 11 1 09l 

e.g., MEAP, ACT, PSAT, 
SAT. etc. 


























Other achievement and/or 
Competency Test Scores 


























Certificates for Achievement 
and/or participation 


























Letter of Recommendation 
from a teacher, counselor, 
principal, etc. 


























Performance appraisal from 
a teacher for a related class 


























Vocational evaluation from 
the Skill Center 


























DAT. GATB, CAPS 



























NON-SCHOOL HECOFOS } A1. 


A2. 


A3. 


A4. 


A5. 


A6. 


A7. 


A8. 


A9. 


A10. 


All. 


A12 


Charts and/or graphs 
created for a report, class 
project etc. 


























Written description, pictures 
or videotape of a 
completed project requiring 
these skills 


























Tape of a speech given for 
any event (e.g. class, award 
banquet 


























Valid Driver's License 


























License to operate 
machinery, equipment, or 
commercial vehicles 


























Copy of your checking 
and/or saving account 
ledger 


























Copy of tax return you 
prepared 


























Letters and job evaluations 
from past/present 
employers 


























Club activity (e.g. math) 
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Teamwork Skills help develop your ability to work cooperatively with a group. They include communicating, 
being responsive, being able to contribute, and being a leader. 



Communicating Responsive 



Tl . Listen to other 
group members* 












T3. Work in changing settings and 
with people of differing 
backgrounds* 












T2. Express ideas to 
other group 
members* 












T4. Be sensitive to the group 
members' ideas and views* 
























T5. Be willing to compromise to 
best accomplish the goal* 













































































































Contributing Leadership 



T6. Actively participate in a 
group* 












T8. Be a leader or follower 
to best accomplish the 
goal* 












T7. Know the group's rul . i 
and values* 

















































































































































* These eoipkiyabiiity skills are necessary to help prepare you to enter the work force or obtain more education. 

FIGURE 8-3 



Source: Michigan Employability Skills Portfolio, Copyright© 1993, Michigan State Board of Education 
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EXAMPLES OF THINGS TO PUT IN MY PORTFOLIO 

Teamwork Skills 



SCrKXX. ncK^UnUb 


n. 




io. 


14. 




TIC 


-r—r 
1 /. 


Io. 


TO 


Tin 

i lU. 


T1 1 
III. 


TIO 


Transcripts w/ class 
descriptions and grades for 
classes using teams and 
cooperation 


























Transcripts from different 
schools attended 


























Letter of Recommendation 
from a teacher, counselor, 
principal, etc. 


























Performance Appraisal from a 
teacher for a related class 


























Le^er describing team work 
contribution from coach, class 
advisor or sponsor 



























NOf^SCHOOL SAMPLES 


T1. 


T2. 


T3. 


T4. 


T5. 


T6. 


T7. 


TB. 


T9. 


T10. 


T11. 


T12. 


Documentation of 
membership in various 
organizations 






1 




















Documentation of leadership 
effectiveness in various 
organizations 


























Job performance 
Appraisal/Work Habits Report 
highlighting group skills (e.g. 
camp counseling) 


























Letter of Recommendation 
from employer, counselor, 
teacher, etc. 


























Awards, Certificates, and/or 
Letters of Appreciation 


























Training Certificates, include 
evidence of effective use of 
skill (e.g. Peer Counseling) 


























Copy of Employment History 


























Tapes of a speech or 
discussion 


























Pictures of team effort 


























Letters 
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Michigan Personal Management Skills Portfolio 



Personal Management Skills help you develop responsibility and dependability. They include setting and 
accomplishing goals, organization, flexibility, and developing career plans. 



RespoDsjibnity 



Organization 



PMl. Attend school/work 
daily and on time* 






J 






PM4. Pay attention to 
details* 












PM2. Meet school/work 
deadlines* 












PM5. Follow written 
instructions and 
directions* 












PM3. Demonstrate self- 
control* 












PM6. Follow oral 
instructions and 
directions* 













































































































Fleaability 



Career Development 



PW. Learn now skills* 












PM10. Know personal 
strengths and weakness* 












PM8. Identify and suggest 
new ways to get the job 
done* 












PMll. Develop career 
plans* 












PM9. Work without 
supervioion* 

























































































































* These employability skills are necessary to help prepare you to enter the work force or obtain more 
education. 



FIGURE 8-5 



Source: Michigan Employability Skills Portfolio: Copyright© 1993. Michigan State Board of Education 
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EXAMPLES OF THINGS TO PUT IN MY PORTFOLIO 

Personal Management Skills 



SCHOOL SAMPLES 


PM1 


PM2 


PM3 


PM4 


PM5 


PM6 


PM7 


PMd 


PM8 


PM10 


PM11 


PM12 


Tmntcripte w/cI«m 
d««criptioot and gnidM for 
r*l*t0d cl«M*f 


























Voc«tion«J Evaluation from 
Skill C«nt»f 


























Lattof of r«comm«ndaljon 
from a tMch«r, counaeiof, 
principal, etc. 


























Pafformanca Appraiaal from 
a taachtf for a raiatad clasa 


























and/or pajticipaik>n 


























Educationai D«v»lopm*nt 
Plan 


























Achi«v©fflant Ta«t Scofwi 


























Att«ndanc« Record 


























Oltdpllnary Record 



























No»facHooL necoRDs 


PM1 


PW2 


PM3 


PM4 


PM5 


PM8 


PW7 


PMB 


PMO 


PM10 


PM11 




Intereet [nventoriea (e.g. 

MOS. COPS, Strong 
Holland. COM) 


























Independent study project 


























Job performance 
appraisal arKl/or Work 
HabltB Report 
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FIGURE 8-6 

Source: Michigan Employabllity Skills Portfolio: Copyright © 1993, Michigan State Board of Education 
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Portfolios - Chapter 8 



«3r Rgure 8-7 (next page) shows how another school system designed 
the constructs for their portfolio project. 

The California Department of Education had Far West Laboratory develop 
and field test portfolios as part of the Career-Technical Assessment Project 
(C-TAP). In the extracted illustration included here, a grid format 
summarizes the expectations to be met during the school year and 
collected into a portfolio. 1"he skill areas to be targeted are listed vertically 
and the work designated to demonstrate the learning goals is shown 
horizontally. 
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Skills Demonstrated In Portfolio Entries 
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FIGURE 8-7 



Source: Caroor-Technlcal Assessment Project Portfolio ■ p. 26, Teacher's Guidebook 
California Department of Education and Far West Laboratory 
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HOW to Choose What Demonstration Data is Needed 



You have decided on the basic format for a portfolio that reflects 
established goals students will demonstrate they have achieved. When you 
designed the portfolio, you also determined which instruments or 
performance tasks could serve to show the skills you established as 
learning goals. 



Now, through a collaborative decision making process, the student and the 
teacher select from the options the particular "proofs" that best suit each 
student's strengths and interests. 



Figures 8-8 through 8-10 show guidelines that are given to the 
students when they are constructing their Michigan Employability 
Skills Portfolio. The targeted skill area is explained to the students, 
with examples of exactly what is meant by the definition. Then a 
listing of suggestions is given for what is "good evidence" and "poor 
evidence" of having the targeted skill. The students choose from the 
tasks listed as good evidence the ones that they think are best for 
them. 



In deciding which examples to extract and include as illustrations in this 
Handbook, we selected three of the areas that teachers often find the most 
difficult to assess. 



«y Rgure 8-8 (next page) is "Problem Solving" 

«3r Figure 8-9 (page 21) is "Flexibility and Initiative" 

«y Figure 8-10 (page 22) is "Contributing As A Team Member" 

All are employability skills that teachers have not traditionally developed and 
assessed in the classroom, and orientation on how to incorporate them into 
the curriculum may be needed. 
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Problem Solving 



Benchmark Problems have many solutions. Problem solvinq requires identifying the goal, 

determining the -esources available, considering the alternatives, and ihen 
deciding which alternatives to act upon. Seeking out information rrom other 
sources (the library, for example), or using other creative processes are often 
appropriate strategies. Troubleshooting, evaluating a failed plan, and learning 
from the experience are also important to the problem-solving process. Some 
specific skills include: 

♦ recognizing and defining problems ♦ selecting the best solutions 

among alternatives 

♦ determining the sources of the 

problem ♦ carrying out decisions and 

evaluating 

♦ finding new and creative ways to 

solve the problem ♦ assessing risk, implementing 

course of action, and evaluating 
results. 



Good EviderK:e The exhibits below, v;hen used with a good written summary, provide evidence 

that you have problem-solving skills. For reviewer approval, you must have 
enough good examples to meet most of the benchmark requirements stated 
above. 



♦ the results from a database 
which you developed, along with 
written summary 

♦ a diagram of a system which 
identifies resource issues and 
possible solutions 

♦ a list of references you looked up 
in the library catalog, with a 
description of search techniques 



♦ a project photo with attached 
descriptions 

♦ a debate av/ard with description 

♦ a project report that shows all of 
the above elements of problem 
solving process 

♦ a written interpretation of data you 
collected to solve a problem 



Poor Evidence The examples below are exhibits which do not provide evidence that you have 

problem-solving skills. These examples are provided to help you understand the 
difference between an exhibit that would meet reviewer approval and those that 
would not. 

♦ a list of references with no ex- ♦ a list of books read 
planation of search strategies 

♦ a test on the use of library skills 
without explanation 



Figure 8-8 



Source: Copyright © 1993, Michigan State Board of Education 
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Flexibility & Initiative 



Benchmark Modem workplaces require constant change and improvement People must 

continuously adapt to these changes. Flexibility means changing your job, goals, or 
plans in order to accomplish a goal. Initiative means doing something original thai is 
needed without being told to do it. Flexibility and initiative may require people to think, 
act, and learn to behave in ne.v ways. Some specific skills include: 



♦ seeking opportunities to update 
your own skills 

♦ participating in continuing 
education and training 

♦ observing, considering, and 
describing better ways to get a 
job done 

♦ persisting in completing training 
on a task until your goal is 
accomplished 



♦ maintaining high performance 
standards without supervision 

♦ going beyond what is askbd to 
improve your project 

♦ recognizing a need or 
opportunity on your own and 
taking appropriate actions 

♦ accepting new or changed 
responsibilities. 



Gkxxl Evidence The exhibits below, when used with a good written summary, provide evidence 

that you have skills and behavior for flexibility and initiative. For reviewer 
approval, you must have enough good examples to meet most of the benchmark 
requirements stated above. 



♦ Completed assignment that goes 
beyond the original assignment, 
with documentation 

♦ output from a computer 
application you use to improve 
your work, with description 



♦ a craft or a project you started 
on your own, with photos and 
written documentation 

♦ written feedback from a teacher, 
scout leader, or employer that 
describes your initiative 



♦ a project photo with attached 
descriptions 



♦ a photo of a flower or vegetable 
garden you planned and 
developed v;ith descriptions 

♦ an award for suggesting a good 
idea that contributes to the 
improvement of a project 



Poor Evidence The exhibits below do not provide sufficient evidence that you have skills and 

behavior for flexibility and initiative. These examples are provided to help you 
understand the difference between an exhibit that would meet reviewer approval 
and those that would not. 



learning to use a word processor ♦ a grade you achieved in a 

(if it was required as part of a required course without summary 

course) 



Figure 8-9 
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Contributing 



Benchmark An effective team needs all of its members to contribute their unique skills and talents. 

Contributing memi^ers must be loyal to the group and what it is trying to accomplish, 
tt is important to follow the team's rules and procedures so that the group's plans and 
goals can be met, but a person cannot blindly follow a group's efforts if he or she 
feels that the group is wrong. A person must first be loyal to himself or herself and 
must possess a high degree of honesty and integrity. If the team's plans and goals 
are consistent with his or her own, contributing is possible. Some specific skills 
include: 



♦ understanding and contributing 
to the mission of the team 

♦ representing the team to others 
that assist in the mission 



♦ demonstrating loyalty to the team 
and showing commitment to the 
group's growth and improvement 



♦ helping the team identify goals 
and norms and contributing to 
achieving them 

♦ making and following a set of 
rules and procedures that will 
contribute to the mission 

♦ helping the team develop to 
meet needs in the future 



Good Evidence The exhibits below, when used with a good written summary, provide evidence 

that you have skills and behavior for contributing to the team. For reviewer 
approval, you must have enough good examples to meet most of the benchmark 
requirements stated above. 



♦ a copy of the student conduct 
code that your student council 
committee developed with 
summary of your contribution 

♦ a letter of recommendation by 
teacher, counselor, employer or 
volunteer supervisor, scout 
leader, or coach 

♦ a copy of a group project or photo 
with an explanation of your role 



♦ a performance rating by teacher, 
counselor, employer or volunteer 
supervisor, scout leader, or coach 

♦ an article in a student newspaper 
you wrote about your team's 
activities with a summary of your 
goals for article 

♦ a cast list from a play that you 
were understudy, showing how 
you supported the cast 



Poor Evidence The exhibits below do not provide sufficient evidence that you have skills and 

behavior to contribute to the team. These examples are provided to help you 
understand the difference between an exhibit that would meet reviewer approval 
and those that would not. 



♦ a picture of yourself on a team 
without summary 

♦ a good conduct grade on class 
report card, without summary 



♦ a copy of a mission without 
summary of your contribution 

♦ membership certificates without 
summary 



Figure 8-10 
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«3P Figure 8-1 1 (next page) contains the steps for collecting work into 
a portfolio given to the students at the Rindge School of Technical Arts 
(RSTA) in Cambridge, Massachusetts. 

In 1991, RSTA initiated "CityWorks©" a new vocational education 
curriculum especially designed to implement the Perkins Act 
mandate for preparing students for "all aspects of the industry". 
Compiling a portfolio is a cornerstone of the students' work in the 
integrative course study at RSTA. 
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Putting Together Your Portfolio 



Steps to Follow: 


1. 


REVIEW CITYWORKS GOALS (#s 1-6 listed on Record of 
Accomplishment sheet). 


2. 


CaATHEH ALL rUHTrULIU MAItnlAL (Keter tO Uutiine OT 
Year provided and organize sequentially). 


3. 


COMPLETE STUDENT RESUME AND RECORD OF 
ACCOMPLISHMENT FORMS. 


4. 


SELECT SAMPLES OF WORK TO ILLUSTRATE 
ACHIEVEMENT OF 4 OF THE CITYWORKS GOALS. 

(More than 4 goals may be selected and more than 
one sample may be included for each goal). 


5. 


PREPARE SAMPLES OF WORK FOR INCLUSION IN 
rUnlrULlu. oampiGS can inciuQG. pornoiio rGcora lorms, 
video tapes; computer discs etc.. All work samples should 
be preceded Dy a Diue organizing page wnicn inaicaies 
what the sample is and which goal it meets. 

**lf you know that you did something during the year 
but don't have or can't find, a record for it, you can 
make your own record by writing about it! There will 
also be blank record forms available. 


6. 


FILL IN TABLE OF CONTENTS. 


7. 


PUNCH HOLES IN PAPER AND INSERT IN BINDER. 


8. 


MAKE PORTFOLIO COVER FOR FRONT OF BINDER. 



Figure 8-1 1 

Source: CityWorksc 1993 
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HOW to Collect the Necessary Data 



Now, you have to create a system to record and monitor the frequency of 
data collection. 



Once it has been decided what will be going into the portfolio of each 
student, you have to plan and keep track of when all the different tasks are 
to be done. Again, this should be a collaborative decision making process 
between the teacher and the student. An important "plus" of portfolios is 
that they give students a sense of ownership and responsibility since they 
are participating in setting their goals and timetables. 



Organizing a schedule can be a very time consuming 
task in itself. In fact, many teachers hesitate about 
undertaking portfolios because they just can't see 
where they will find all the time they envision being 
involved in the process. 

However, the "it will take too much time" fear lessens after the initial stages 
of portfolio assessment have been taken. 

Once the portfolio process is underway, it acquires an energy of its own. 
With more experience, the classroom environment becomies an 
encouragement for students to be increasingly independent and 
responsible for assessing their own progress. 
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To reduce the time involved in implementing a portfolio assessment 
process, organize the goals well and within a structured time frame around 
the following factors: 

1) Make students responsible for collecting information on a regula" 
basis. 



2) Make the data collection part of periodic, even daily, instructional 
routines. 



3) List on a portfolio analysis form the specific items that will go into the 
pcrtfolio. 

4) Share responsibilities of data collection and interpretation witii other 
school staff so that individual teachers don't become ovenwhelmed 
by the process. 

5) Create planning/conference times for teachers and other staff 
members involved in portfolio development. 



^ In Figures 8-12 through 8-14, we show examples of different ways 
educators have addressed these concerns and developed effective 
answers. 



Bar Rgure 8-12 (page 27) illustrates how students using the C-TAP 
portfolios keep track of their responsibilities and schedules. It is a 
simple outiined check list that is kept in the front of their folders. 
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Portfolio Sign-off Sheet Student: 

Keep this form at the front of your Class: 

working folder. Check off each 

item as it is completed. Some Date: 

pieces need to be reviewed or 

signed off by your teacher. Teacher: 

Space is provided for your 

teacher's initials. 

Check off when complete: 

Table of Contents 
Letter of Introduction 
Ql Career Development Package 

^ Application for employment or college 
Letter of Recommendation 
R6sum6 
Work Samples 

Q Work Sample #1 and caption 
Work Sample #2 and caption 
Work Sample #3 and caption 

□ Work Sample #4 and caption 
Ql Research Write-up 

□ Outline (with teacher comments) 
Draft (with teacher comments) 

Q Final write-up 
□ Supervised Practical Experience Evaluatir.i 

Figure 8-12 



Source: Caroor-Technical Assessment Project, Page 39 in Teachers Guidebook 
California Department of Education and Far West Laboratory 
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isr Rgure 8-13 (next page) shows how the teachers using the C-TAP 
portfolio keep a progress check on each student's work. 

Rgure 8-14 (page 30) is the "Learner Profile and Planner" ihat 
students use as a periodic progress check while they are 
constructing a Michigan Employability Skills Portfolio, 



Page 28 
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Teachers: 

Class: 

Semester: 

Portfolio Progress Check 

Use this sheet to record due dates, completion dates, status and/or scores for each portfolio section or entry. 



Student 


Introduction 

• Table of Contents 

• Letter of 
Introduction 


Careef Deveiopment 
Package 

• Application 

• Recommendation 

• R6sum6 


Work Samples 

• 4 Samples 

• Captions 


Research Write4Jp 

• Outline 

• First draft 

• Final report 


Supervised Practical 
Evaluation 


Rna! Port 























































































Figure 8-13 



Source: Career-Technical Assessment Project Portfolio - p.21 . Teacher Guidebook O 1 T 

California Dep/^irlment of Education and Far West Laboratory ^ 1 O 
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Name: 
Date: 



Learner Profile & Planner 

Not Not Shows 

Available Explained Skills 

Academic 

1. Communication | | j | j j 

2. Mathematics | | | j j j 

3. Science and Technology | j | j j j 

4. Problem Solving | j | [ | | 
Personal Management 

5. Responsibility | | | [ | [ 

6. Organization | | | | j j 

7. Flexibility | | j | | | 

8. Career Development | | j j j | 
Teamwork 

9. Communicating j j | [ [ | 

10. Responsiveness | j | | | | 

1 1 . Contributing | j | | | | 

12. Leadership/Followership | | | [ j | 



What 1 need to do: 





















Figure 8-14 

Source: ©Copyright 1993. Michigan State Board of Education. All Rights Reserved. 
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HOW to Analyze the Portfolio Conterrts 



The portfolio has been designed, the "evidence" of achieving determined 
skills has been collected, and now how is it all interpreted? By what 
standards will the student's entries be judged? 



Performance criteria must be established before collecting any student 
data— that way it will be clear from the start what is expected for the entry 
to be considered an "achievement" and to what degree. Criteria that are 
dearly and collectively established make it easier for the students and the 
teachers to assess each piece of work consistently. Knowing what is 
expected translates into work being judged as: 

• Exceeds Standards 

• Meets Standards 

• Approaches Standards 

• Well Below Standards 

The students can thereby see their own progress and know exactly where 
they stand; what they still need to work on and what they have achieved. 

Some school systems follow the SCANS guidelines that established five 
levels for assessing skills and competencies, going from 

Rudimentary -> 

to Basic -> 

to Intermediate -> 

to Adept -> 

to Advanced. 

Others have condensed the assessment levels to three, usually classified 
as "Advanced", "Proficienf, and "Basic". 

is- Figure 8-15 (next page) shows an assessment scale that is used to 
evaluate the work included in the C-TAP portfolios. 
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Portfolios - Chapter 8 











HOW to Use the Results 



When you made the initial decision about the purpose of your portfolio, you 
were already determining how it would be used. The learning process that 
you want it to document leads to a product that will interest a certain public. 



• Did you want your students to have something of practical value to 
show potential employers? 

• Did you want your students to have a collection of class work that 
proves competence in your subject matter and serves as a basis for 
a course grade? 

• Does your school district or community college want a source of 
accountability for gauging programs' effectiveness? 



Ideally, a portfolio can be used to fill all of these diverse needs and many 
more. 

ra- In pages 4-5 of this chapter, 

we presented the many reasons why teachers in California are developing 
and using portfolios in their classrooms. The inclusive list shows that 
portfolios can be used for access assessment and/or on-going assessment 
and/or exit assessment. 



Access Assessment: When a student applies for a job or to enter a 

new program, a portfolio will show if and how the 
applicant can meet the required qualifications. 
Much more than the score from a standardized 
pencil and paper test, a portfolio gives 
indications of the depth and scope of a 
candidate's skills and preparation. 
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Especially in the case of LEP students, a true 
demonstration of skills is many times distorted 
by the language used in standardized tests, A 
portfolio can be an "international language" by 
showing what has been actively achieved. 



On-going Assessment: As a long range undertaking, a portfolio gives 

the teacher, as well as a student, continual 
feedback about: 

• the progress being made; 

• the need for further instruction; 

• the effectiveness of instruction; 

• the accuracy of curriculum objectives; and 

• the achievement of educational goals. 



Exit Assessment: Since a portfolio is a product, as well as a 

process, it can be evaluated to see if it can be 
considered a 'final" product. Key "exit" 
questions can be answered by looking through 
a portfolio and seeing: 

• Does it demonstrate that a student has 
fulfilled the objectives of a classroom or 
a program, and is ready to move on to 
the next one? 

• Has the student mastered set com- 
petencies at a prescribed standard of 
achievement, and therefore can be 
considered qualified in that area? 
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HOW Portfolios Can Be Used for Other 
Educational/Career Related Purposes 



The most common use of portfolios is for students to have tangible proof 
of developed integrated skills to "showcase" their strengths to other people. 
The "exhibition" factor in many cases is a key for motivating students to 
make renewed attempts to produce their best work. 

However, the American School Counselor Association (ASCA) published 
last year "a personal Planning Portfolio for career development" called 
Get A Life<s>. Students use it almost as a structured persona! diary to guide 
their reflections on important areas of life development and the value of 
education to future life styles and career choices. It gives students the 
outline for seeking, analyzing and recording their own guidance information 
insights. 

Creating a portfolio that is a personal career planner makes the student the 
central player in reaching vocational/life decisions through assessment 
filtered through a metacognitive process. The student takes the lead in an 
assessment/advisor area usually reserved, conserved and isolated in 
biannual meetings with a counselor. 

Why are we giving this strategy a section in a Handbook directed to 
teachers? Because many teachers might not realize the key role they 
themselves play in influencing students' decisions when it comes to 
choosing a course in school or a path in life. 

Consider the resuits of two studies on where a student seeks or can expect 
to find vocational or educational/related guidance. 

Chinatown Resources, Inc., noted a curious phenomenon occurring among 
LEP students who were newcomers to the California educational system. 
Most had a very haphazard understanding of opportunities available in the 
local schools. Compounded with the fact that many were also newcomers 
to the United States and ill-informed about the American world of work, LEP 
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students were very likely to have misconceptions about vocational 
education needs and possibilities. 

Despite the shared problems in acculturation, LEP students preferred to rely 
on each other for guidance information. A survey found that they most 
frequently turned to peers for information regarding school or work choices. 
(Chinatown Resources Development Center, 1985). The other possibilities 
listed in terms of trust and availability: 

• Second most frequent source of information: Teachers 

• Last and seldom mentioned source: Counselors 



Counselors were also a scant source of guidance for occupational choice 
or job placement in a study done by ETS (ETS Policy Information Center, 
1990). A statistical analysis of how high school guidance counselors 
directed their attention and time yielded the following graphic results: 



How High School Counselors Spend More than 30% of Their Time 



Choice of High 
School Courses 

College Admissions 
and Selection 

Attendance and 
DisopTine Problems 

Occtipaiional Choice 
Of Career Planning 

Job Placement 



34 



30 



11 



27 



25 



0% 10% 20% 30% 

Percent of Counselors 



40% 
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The students themselves, followed by teachers, emerge from the two 
reports as the prime sources of information that influence choices in school 
and in life. A personal planning portfolio, such as Get A Life ©, gives a 
structured guideline to the principal players for assembling and assessing 
the information into a more effective basis for decisions. 



HOW This Type of Portfolio Is Organized 



Having a different purpose means that this type of portfolio also has a 
different format. 

Get A Life © is a slim booklet with tabbed dividers marking the sections of 
printed pages to be filled in by the student. There are four areas of 
information to be collected and analyzed: 



Self Knowledge (Personal qualities, data, skills, documented 
activities of competence, etc.) 



Life Roles (Past, present and future roles at home, in school, at 
work) 

Educational Development (Looking at job and basic life 
"competencies" through educational experiences) 



• Career Exploration and Planning ( Specific competencies and steps 
involved in preparing to look for a job or furthering a career) 

Tucked into a flap at the end of the booklet is a foldout sheet on which the 
student compiles and summarizes the collected information. We show it 
here in reduced format as an overview of the self-assessment steps 
students are asked to discuss with ^'advisors, teachers, parents, employers 
and other supportive people...". 

Figure 8-16 (pages 38-39): "Get a Life: My Personal Career Plan"© 
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GET A LIFE 

MY PERSONAL CAREER PU\N 
On these pages, you are asked to reflect on several important pieces of the Career Decision-Making process. Someone in your school 
will assist you in deciding what kinds of entries might be helpful 



SELF-KNOWLEDGE - The most important things 1 need to remember about my unique seff as 1 make career decisions are: 
1. 

2. 

3. 



AWARENESS OF THE WORLD OF WORK - Entire worksheets and surveys will probably be used to assist you in understanding the World 
of Work. Include these in your portfolio. In this space, write about the most Important lessons you need to remember about the World of Work. 



CAREER ASSESSMENT INFORMATION - Key information from assessments that should enter into my career decisions: 



LINKING SELF-KNOWLEDGE WITH THE WORLD OF WORK- Occupations I have identified where 1 can use my personal 
strengths, interest, and values: 



CAREER EXPLORATION - Activities that have helped my experience work and to sort out the things 1 like or dislike about various career 
options: 



CAREER OPTIONS - Career choices that interest me are: 



1. 
2. 
3. 
4. 
5. 



CAREER DECISION - Having given considerable thought to career decision-making, at this point in my life I am interested in pursuing the 
following career options: 



The Key Factors in this decision are: 



Figure 8-1 6 



CAREER PLAN ~ i have worked out a plan for how i will prepare for my future. The basic steps are listed here: 
Bmore detailed plan may be found 



TRAINING - To obtain the training 1 need to be successful in my chosen career, 1 will take the following steps. 



TRAINING DECISION ~ I have decided that the best training/preparation for my future is: 



The Key Factors in this decision are: 



JOB-SEEKING SKILLS - I have completed the following items to help me as I look for a job. 



Resume 



j I Sample Cover Letter 
I I 



I Identify Resources for Locating Employment 



j I List of References 
I I 



Some important things for me to remember as I look for a job: 



Interview Practice 



Sample Job Application 



Other: 



EMPLOYABELITY SKILLS - While I pursue my career, I must develop the knowledge, skills, and attitudes that will help me keep a job or help 
me find new jobs in a changing job market (e.g., promptness, dependability, good attendance, work skills). Feedback from adurts and friends suggest 
my strongest employability skills are: 



WORDS OF WISDOM - Helpful advice and counseling I have received while exploring and planning for my career. 

I 
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Two other specific pages of the portfolio merit reproduction in this 
Handbook. They exemplify a focus that can both open up areas for LEP 
students, and also orient work-related experiences for students in vocational 
training. 



Rgure 8-17 (next page) is a page of guidelines given for students 
to better understand themselves and what goes into the "Life Roles" 
they play. It is encouraging to see that students are asked to think 
about the contributions that their "cultural heritage", and family's 
dreams have made to expanding their lives. 



Figure 8-18 (page 42) is a clear way of seeing at a glance how 
prepared a student is for the world of work. 
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RESULTS OF CAREER ASSESSMENT 



NAME OF ASSESSMENT 



DATE 



RESULTS 



4. 



5. 



PRE-EMPLOYMENT EXPERIENCES 



I I Talked with parents or adults about career plans 

j j Participated in extracurricular activities 

j I Participated in coipmunity organizations 

I I Explored career information in the Career Center 

I I Held leadership positions or offices 

I I Served as a volunteer 

I I Took a field trip related to a career 

I I Participated in a similar interview 

□ 



Chose references 



I j Took part in a career day 

I I Held part-time or summer jobs 

I I Heard speakers (including college rep.) 

□ 
□ 

j j Filled out applications 

I j Wrote for information on careers 

I j Participated in an actual interview 

I I Completed a resume 



Interviewed (a) workerCs) 
Observed or shadowed a worker 



WORK EXPERIENCE (Include Volunteer & Part-time Work) 



EMPLOYER 



V/HEN 



TYPE OF WORK 



SKILLS 



Figure 8-1 8 



Source: Get a Llfoo: ASCA 
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WHERE? 



Educational Institutions That Have Developed 
and Used Portfolios 



Amakigchik Elementary School and 
Chaputnguak High School 
Lower Kuskokwin School District 
Box 50 

Chefomak, AK 99561 
(907) 867^700 
Contact: Sandy Fuller 

Last year this school began a language arts portfolio project. Grades one 
through twelve are using portfolios as an assessment tool. At the end of 
the year all students give a presentation of their portfolios to a portfolio 
analysis team. The teams are organized to consist of a teacher, principal, 
parent, and school board member. The portfolio content reflects skills and 
final products outlined in the Lower Kuskokwin School District's curriculum 
guide. The students in the school are bilingual Yup'ik and English speakers. 



American School Counselor Association 
5999 Stevenson Avenue 
Alexandria, VA 22304 
(703) 823-98C0, Ext. 388 

Creators of Gef a Life<s> 
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Bell Mutticuttural High School 
3145 Hiatt Place, N.W. 
Washington, D.C. 20010 
(202) 673-3551 



Escondido Union High School District 
Valley High School 
240 S. Maple 
Escondido, CA 92025 
Contact: Robert Clay, Principal 

Continuation school, grades 10-12, Senior Portfolios. Students build 
portfolios with advisory teachers. The portfolios contain graduation 
documents, certificates of program completion and of recognition, resumes, 
vocational profiles, community services/activities, attendance record, and 
samples of student academic work. 



Far West Regional Laboratory (FWL) 
730 Hanison Street 
San Francisco, CaJifomia 941 07 
(415) 565-3000 

For information on C-TAP Portfolio. 

Under contract with the California Department of Education, FWL is 
developing and field-testing a new system to assess and certify vocational 
education students in California high schools, adult education and Regional 
Occupational Centers/Programs. A student portfolio will play a major part 
in the assessment system under development. Pilot test for the 1991-92 
school year included 800 students in 40 vocational education classes 
across the state. 



Madison Asea Technical College 
Altemative Learning Division 
21 1 N. Carroll Street 
Madison, Wl 53704 
Contact: Karen Johnson Kretschmann 
Betty Hayes 

Adult Basic Education - Pilot project which began 9/93 will develop an 
alternative assessment model for subject area reading/writing and math 
which is competency-based and which will provide more comprehensive 
and useful student outcome data. 
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Michigan State Board of Education 
P.O. Box 30008, 
Lansing, Michigan 48909 

For information on Michigan's Employability Skills Portfolio© 



Minuteman Regional Vocational Technical School District 
758 Marrett Road 
Lexington, MA 02173 
(617) 861-6500 

Contact: Sebastian R. Paquette (exL 325) 

Portfolio concept introduced in full use by 9th and 10th grade classes in 93- 
94 school year to be expanded to 9th-Post Graduate by 95-96, The student 
owned, instructor assisted portfolio displays academic and vocational 
achievements as well as other documentation (hard copy and disk format). 
CompuTurs, scanners, cameras and laser printers are currently used to 
assist documentation. Future plans include CD ROM. 



NASSAU Technological Center 
1 1 96 Prospect Avenue 
Westbury, New York 1 1590 
(516) 997-5410 

The Bilingual Language Support Program is developing an innovative 
assessment process for LEP students in vocational education programs. 



Prince William County Public Schools 
Office of Assessment and Evaluation 
P.O. Box 389 
Manassas, VA 22110 
Contact: J. Michael O'Malley 

Prince William County developed performance assessment measures to 
accompany curriculum revision in language arts, math, science, and career 
and technical education during 1992-93. The assessment measures are an 
integral component of the curriculum revisions, which include upgraded 
outcomes and standards in each area. Performance assessments will be 
used along with portfolios for instruction planning on all students, and will 
be sampled for building-level and district-wide accountability. The 
accountability measures will be at benchmark grades such as grades 4,7, 
and 11. Central scoring of the measures used in accountability (a 1-hr on- 
demand performance measure plus the portfolio) will ensure acceptable 
inter-rater reliability. The county will collect only an "assessment" portfolio 
containing writing samples of project information, while teachers will 
maintain a forking" portfolio for use in instructional planning. 
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Rindge School of Technical Arts 
459 Broadway 

Cambridge, Massachusetts 02138 
(617) 349-6753 

Creators of CityWorks^ curriculum. 

University of Northern Colorado 

College of Education 

Teacher Education Center 

103 McKee Hall 

Greeley, CO 80639 

(303) 351-2430 or 1-800-235-8255 

Contact: Linda Brookhart, Doug Maclsaac 

Students begin the development of a professional teaching portfolio 
program upon entry into a professional teacher education program. The 
teaching portfolio reflects a synthesis of the pre-service teacher's beliefs, 
theories, competencies, understandings, and experiences. Approximately 
700-1000 students a year at various program levels are involved in 
constructing teaching portfolios. 



Van Buren Intermediate School District (VBISD) 
250 South Street 
Lawrence, Ml 49064 
Contact: Don Olendorf 

The VBISD has developed a curriculum for teaching students how to 
develop portfolios. This student managed process enables high school 
students to plan, organize, document, and summarize their work in core 
curriculum areas with an employability skills overlay. 



York College Community Learning Center 
94-43 159th Street 
Jamaica. NY 11451 
Contact: Jane Mackiiiop 

York College Learning Center (City University of New York) serves 
approximately 2,500 adult learners per year with a range of diverse services 
including ESL, Adult Basic Education, intergenerational literacy, GEO 
preparation, career counseling, job training and referral and child care. 
Portfolios are used to assess change and progress from the moment of 
initial assessment until a student leaves the program. 
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Resource Centers and Other Organizational Networks 
to Contact About Using Portfolios 





Northwest Evaluation Association (NWEA) 
5 Centerpointe Drive 
Suite 100 

Lake Oswego, OR 97035 
(503) 624-1951 
Contact: Allen Olson 

The association publishes Assessment Alternatives Newsletter three times 
a year. The association's goal is to support an information network for 
educators interested in portfolios and portfolio assessment. 



Northwest Regional Educational Laboratory (NREL): Assessment Center 
101 S. W. Main Street 
Suite 500 

Portland, OR 97204 
(503) 275-9562 
Contact: Dr. Judy Arter 

NREL has a clearinghouse which disseminates a newsletter, reports on 
portfolio projects, and annotated bibliographies of articles and papers on 
portfolios. 

Portfolio Interest Group Program 

Cooperative Research and Extension Services for Schools (CRESS) 

Division of Education 

University of California 

Davis, CA 95616 

(916) 752-8470 

Contact: Jim Hahn 

The CRESS Center at UC Davis is interested in helping teachers form 
portfolio interest groups in order to '-.prove instruction and assessment in 
different subject areas. The *^enter's goal is to assist teachers in their 
efforts to design, try out, critique and redesign programs for using portfolios 
of student work in their own classrooms. 
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Portfolio Assessment Clearinghouse 
University of California, San Diego 
Teacher Education Program 
9500 Gilman Drive - 0070 
La Jolla, CA 92093-0070 
Publishers of Portfolio News© 



Writter 'nformation on Portfolios: 



Crandall, Jo Ann, Directions in Vocational Education for LEP Students and 
Adults . (1985). Columbus, Ohio: National Center for Research In Vocational 
Education. 

DeFina, A., Portfolio Assessment: Getting Started . (1992). New York: 
Scholastic, Inc. 

Duran, R.P. (1989) ."Testing of Unguistic Minorities". In R.L Lynn (Ed), 
Educational Measurement (3rd Ed., pp. 573-587). New York: American 
Council on Education. Macmillan Publishers, Inc. 

Fradd, Sandra H. and Weismantel, M.J., Meeting the Needs of Culturally 
and Linguistically Different Students: A Handbook for Educators. (1989). 
Boston, Massachusetts: Little, Brown, & Company. 

Gardner, Howard, "Assessment in Context: The Alternatiye to Standardized 
Testing", (1992). In Gifford & O'Conner (eds). Changing Assessment 
Boston, MA: Kluwer Academic Publishers. 

Grognet, Allene, "Work Skills and The Limited English Proficient Student", 
(1990). Washington, D.C.: Center for Applied Linguistics. 

Hamayan, E., and Damico, J., (eds.). Limiting Bias in the Assessment of 
Bilingual Students. (1991). Austin, Texas: Pro-Ed. 

Hamayan, Kwiat, & Perman, Assessment of Language Minority Students: A 
Handbook for Education . (1985). Arlington Heights, Illinois: Illinois Resource 
Center. 

Mitchell, Ruth, Testing for Learning. (1992). New York, New York: The Free 
Press. 
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Navarrete, Cecelia et al, Informal Assessment in Educational Evaluation: 
Implications for Bilingual Education Programs , (1990). Washington, D.C: 
National Clearinghouse for Bilingual Education Programs. 

Raizen, Senta, Reforming Education for Work: A Cognitive Science 
Perspective, (1989), Berkeley, California: National Center for Research on 
Vocational Education. 

Resnick, Lauren, "Assessing The Thinking Curriculum: New Tools for 
Education Reform", (1992), In Gifford (ed), Changing Assessment: 
Alternative Views of Aptitude, Achievement and instruction . Boston: Kluwer 
Academic Publishers. 

Shepard, Lorie, "Commentary: What Policy Makers who Mandate Tests 
Should Know About the New Psychology of Intellectual Ability and 
Learning", (1992). In Gifford (ed), Changing Assessments: Alternative Views 
of Aptitude, Achievement, and Instruction . Boston: Kluwer Academic 
Publishers. 

Stiggins, Richard J,, "Revitalizing Classroom Assessment: The Highest 
Instructional Priority", (1988). Phi Delta Kappan. Vol. 69, No. 5. 

Pierce, L.V., and O'Malley, J. M., "Performance and Portfolio Assessment for 
Language Minority Students", Program Information Guide, Series, 9. (1992), 
Washington, D,C.: Department of Education, Office of Bilingual Education 
and Minority Language Affairs, 

Zessoules and Gardner, H., "Authentic Assessment: Beyond the Buzzword 
and into the Classroom", (1991), In Perrone (ed,). Expanding Student 
Assessment. Alexandria, Virginia: Association for Supervision and 
Curriculum Development, 
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PROJECTS 



WHAT? 



The concept of projects is as diverse as the students we seek to serve. 
For example: 

• Some schools in California see projects as condensed "Work 
Samples", illustrations of skills that students have targeted as 
demonstrations of their applied knowledge. 

• A vocational education high school program in Cambridge, 
Massachusetts, has a curriculum based on "Exploratory 
Projects", tangible results from students exploring and 
learning about how their community works. 

• In high schools in Fort Worth, Texas, students do projects as 
a collection of "Artifacts", pieces of proof that show how 
students approached a problem and resolved it in a skilled 
manner. 

Projects frequently form the bulk of a portfolio's contents; in fact, the 
definitions of projects and portfolios overlap in the area of actively 
demonstrating acquired skills. But, projects are more of a short-range 
approach for assessing a student's on-going progress, and therefore may 
be more manageable methods for many practitioners and students to develop 
than full-scale portfolios. 
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WHAT Are the Advantages of Using Projects as A 
Teaching/Assessment Tool? 



The advantages of having projects form an integral part of the classwork 
are similar to those of portfolios, especially for LEP students. Projects, like 
portfolios, can open the assessment process through: 



• Emphasis on teamwork 

• Inclusion of visual aids to supplement or better explain the 
written parts 

• Offerings of options that span a diverse range of interests and 
skill levels 




WHAT Are Some Examples Of Projects That Teachers 
Have Innovated With LEP Students In Vocational 
Education Programs? 



Repeatedly throughout our work, practitioners expressed an interest in 
knowing what others in the field are doing in the classroom to make 
learning and assessment a more integrated, active process. Projects are 
a popular way of doing that. 



With the purpose of providing a "clearinghouse" look at some of the 
dynamic projects that have been successful, but also with the caution that 
what works in one school setting might not be applicable to all, we include 
a short sampling of possibilities. 
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EXAMPLES OF PROJECTS 



An Electronics Project That Reaches Into the 
Community 



An Electronics course at the Cambridge School in Weston, Massachusetts, 
offers a project that focuses on all aspects of radio and its relationship to 
the Haitian community in the area. It touches not only on how a radio 
functions, but also the history of the radio, Haitian cab drivers who 
communicate via two-way radios, and local Haitian radio shows. 



Cultural Allowances Add to a Project of Designing 
and Furnishing a Home 



Giving a good example of an integrated curriculum approach, the Home 
Economics, Carpentry, Technical Drawing, History, and Art teachers in a 
high school in New Mexico created a project of designing and furnishing a 
Victorian house. With a student population that is largely Hispanic, the 
modification was allowed for several students to do a sophisticated design 
in the traditional adobe Southwestern tradition. (Stasz et.al., 1990). 
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III. An Integrated Bilingual Project Based On Tech Prep 
Materials 



The Tech Prep Applied Mathematics teacher joined with the Language Arts 
Development Director to design a project at Bell Multicultural High School 
in Washington, D.C„ In this vocational training-oriented high school, around 
50% of the student body is Hispanic. The two teachers developed a project 
to have the students in their classes prepare sub-titles in Spanish for the 14 
video materials that accompany the Tech Prep Applied Math course as an 
audio/visual extension of the textbook. The project is presently in progress 
and has aroused the interest of the textbook's publishers. The Texas- 
based company plans to review the finished product, with an eye for 
possible national distribution. 



IV. A Woodworking Project That Makes a Classroonn a 
Studio 



A cabinet maker and teacher in a New York high school has his entire 
classroom decorated with different pieces of furniture made by previous 
students. From the first day of class, everyone can see that high quality 
work is expected of them; work that is to be chosen, designed and made 
by the students themselves. The students start their year by deciding what 
they want to create and then learn the "tools of the trade" by doing so. This 
makes an innovative alternative to the typical, simple and standardized 
approach of instructing students to make bookends or breadboards after 
completing assigned exercises on separate skills, (Little, 1992). 



Page 4 Crosspaths Management Systems, Inc. 



ERIC 



Projects - Chapter 9 



V. A Project that Organizes an International Covered 
Dish Social 



At Brevard Community College in Palm Bay, Florida, instructors in the VESL 
program coordinate a project designed to bring together a diverse student 
body into a cooperative whole. The "International Covered Dish Social" 
assignment has the students organize, plan, and produce relevant tasks 
that also serve as steps towards bridging cultural barriers to learning. 

The assignment is for the class to prepare a dinner of intemational dishes, 
and invite outsiders to attend and enjoy the social. Everyone in the class 
is expected to participate; this includes speaking , listening (with respect), 
and doing. 

After each student has described a favorite native dish, the class decides 
on a menu. They also prepare a list of the people to be invited. They then 
perform a task analysis outlining all the chores that need to be done and 
distribute a task sheet. Volunteers choose from the identified tasks. 

From the many performance tasks generated by this project, the instructors 
and other students can rate the skills involved in cooperativeness, 
participation, and technical accomplishments. 



VL 'Work Samples" Projects Used as Part of Portfolio 
Assessment 



In the portfolios that high school students construct as part of the Career- 
Technical Assessment Project (C-TAP), used in some California schools, 
four Work Samples are included that show mastery of important skills. The 
Saniples are products of ideas in different content areas that the students 
have developed into tangible results. Sample project choices suggested 
by content area to the students are: 
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Business Education: Computer Science and Information 
Systems 



1) Produce a mailable letter that contains merged information 
and also showing the merge codes and a printout of the 
database used. 

2) Produce a newsletter using desktop publishing software. 



B. Industrial and Technotogy Education: Construction 
Technology 



1) Present a complete plan, with jigs and fixtures, to mass 
produce a wood box. 

2) Develop and print out a computer-generated flow chart for a 
construction project. 



C. Health Careers: Hospital Information Sen/ices 



1 ) Complete insurance forms applying all necessary codes using 
information in a patient's chart. 

2) Design a poster showing how electronic equipment is used to 
communicate Interdepartmentally in the acute care facility. 
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D. Home Economics: Child Care Occupations 



1) Design a chart comparing the equipment and safety features 
of the outdoor play areas at three different day care centers. 

2) Create a display of recipes and posters showing children 
working together to make three healthy snacks. 

^ See Page 24 of this Chapter, which illustrates a project in this 
skill area. 



E, Agriculture: Animal Science 



1) Design a livestock facility for a selected species which 
includes the efficient and proper disposal of animal wastes. 

2) Develop a feeding program for a selected species of livestock 
designed to meet the nutritional needs of the animal. 



VIL Using Projects as a Cornerstone in an "All Aspects 
of Industry" Vocational Education High School 
Curriculum 



At the Rindge School of Technical Arts in Cambridge, Massachusetts, 
projects form an essential part of a vocational education curriculum. The 
program, called CityWorks, was specifically designed to implement the 
Perkins Act mandate for preparing for "all aspects of the industry". Over a 
nine week period, the students at this public high school rotate in teams 
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working on projects that they have chosen from integrated offerings in 
4 areas. The Rindge student population is approximately 46% bilingual and 
15% limited English proficient, with Spanish and Haitian Creole being the 
predominant native languages among LEP students. 

NOTE: The 9th grade curriculum, 'CityWorks', is copyrighted by the Rindge School of Technical Arts. 

59r See page 30 of this chapter 



Building a School 
Projects 



"Museum" to House the Completed 



Tie Me Up" 



Learning what it takes to "invent", plan and make 
a workable machine using the tools of the 
welding trade. The end result is a machine with 
moving parts that makes rope. 



"Stop and Go" 



Making a working headlight and tail light 
assembly using the equipment in the auto shop. 
This involves drawing up a wiring diagram and 
hooking up a battery and flasher units. The final 



model has working directional, parking, tail, head and stop lights. 



Building and constructing a working scale 
model of the school's Museum and studio 
spaces. Focuses on the different stages 
involved in building construction by designing 
new studio spaces to house the exhibits [the finished projects] of the 
Museum. 




"Words for Work^' 



Working with students in all of the groups 
making a dictionary of technical and "buzz" 
words for each industrial area represented in 
the school Museum. The completed 
dictionary contains definitions, illustrations and photographs. Each student 
in the class has opportunities to participate in gathering information and 
working on the finished product, and also making the tapes for the talking 
robots. The multilingual learning opportunities in this project are open and 
very exciting for a cross-cultural student body. 



Page 8 



Crosspaths Management SyGtcms, inc. 



ERIC 



Projects - Chapter 9 



B. Creating a New Restaurant for Cambridge 



"Be a General (Contractor)" 



] 



The students are the general 
contractors for the restaurant job, 
figuring out what kinds of building 
— materials to use in the construction 
and what everything will cost. The students find out from City Hall about 
how to pull a permit, and they visit construction sites to get an idea of what 
else contractors do. 




So you want to own or work in a 
restaurant! This project exposes the 
student to the how and why of kitchen 
and menu design. The students decide 
on the location, size and theme of the restaurant; determine the menu items 
and prices, and plan the layout of the kitchen and the equipment needed. 

This project plans the breads and 
desserts for the restaurant, and the 
layout of the eating area. The students 
visit a few restaurants to inten/iew 
owners and pastry chefs to get information about desserts. Are people 
more health conscious these days? Are they eating less? Should there be 
whole grain breads? The students also look at the I: yout and facilities of 
restaurants, and note what they should think about in choosing furniture, 
how they can make sure customers and waiters have room to work or eat 
comfortably. 




"Lights, Sound, Action" 



Lighting and sound play a big part in 
creating a pleasant atmosphere in a 
restaurant, and this project aims to 
design good systems for both aspects. 
The students go over electrical plans for the restaurant, talk to peop's in the 
city and at the electric company about permits, and take trips to sound 
system stores and lighting display rooms to price items. 
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•What a Site" 



The students first have to decide on a location 
for the restaurant. Then they draw a sketch of 
the site and decide where to locate the building 
on the site, and what the building should look 
like to take maximum advantage of the site (how many stories high, what 
shape, what kinds of windows, where, etc.). They create a scale model of 
the restaurant. 



C. Creating a New Auto Body Shop for Cambridge 



"Hard Cash" 



It takes a lot of planning to put together a business 
like an auto body shop. The students in this 



project get advice from people who run body 
shops and then figure out how to make theirs 
better. They figure out all the equipment they need and what it costs. They 
find out about the permits they need. They also make sure that none of the 
other groups working on the rest of the Auto Body Shop Project spend too 
much money! 



"Draw It Out' 



The job here is to plan the layout of the auto body 
shop and then build a scale model of it. They draw 
out the floor plan, deciding where to locate the 
equipment, paint booth, etc., and the waiting area 



for customers. They do instruments lay-outs and detail drawings. 



"Customer is Not Always Right' 



An auto body shop needs 
customers. The first thing the 
students on this project do is 
come up with a name for the shop 
and a logo. Then they make signs, design an ad campaign and come up 
with slogans for the ads. They also work on the customer reception area, 
planning the furniture and interior design. 
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One of the most important things about a body 
shop is the air ventilation system. The students 
figure out what equipment is needed and what 
the requirements are for this kind of system. 
Then they design one for the shop and figure out the sizes of the ducts and 
where to locate them. They make a scale model of the system to put inside 
the model of the shop. 



D, Creating a New Teen Activities Center for Cambridge 



"A Happening Place" 



What makes a good Teen Center 
program? Who usually makes the 
decisions? The students find out about 
how other teen centers run and then 
figure out how to make theirs better. They come up with a finished 
(printed) activities program for their center. 



••Everybody's Turf* 



What makes teenagers feel welcome or 
unwelcome in a Teen Center? What could 
be done to make the student-designed 
Center welcoming to people of all races and 
backgrounds? The students learn about the ways that people get divided 
and figure out a way to bring them back together. This project is another 
good example of how a multicultural student body can be integrated into 
dassroom activities with positive input and gains. 



This Teen Activities Center is going to have 
a woodworking shop where teenagers can 
make things for the Center, or things to 
sell or give away as presents. It is the job 
of this group project to plan the woodworking shop and make a scale 
model of it. 
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"On the Road- 



Some teens have cars that need lots of repairs to 
stay on the road. This Center will have an auto 
repair shop where kids learn basic skills of repair 
and work on their own and their friends' cars. It 
is the students' job to plan the auto repair shop and make a scale model 
of it. 



"A Site for Sore Eyes" 



Once the students have decided on 
where the Teen Activities Center should 
be, they make a sketch of the site. 
Then they figure out how to use the site, 
how to divide up indoor and outdoor space, and what kind of structure will 
work best on that site. They make a three-dimensional site plan, showing 
the results of their decisions. 



"Lay It Out" 



The students design the main lounge-recreation area 
for the Center, first using sketches and then making 
blueprints. They talk with "The Happening Place" 
and the "Everybody's Turf' groups to find out what 
kind of lounge area they want and what they want in it. Then they will 
design it. 
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WHY? 



I hear and I forget; 
I see and I remember; 
i do and I understand. 
-Chinese proverb 



it is very fitting that a main reason for using projects is found through 
translating universal wisdom from another language. 

Projects create a bridge between thinking and doing that can take LEP 
students beyond linguistic barriers that might be blocking a demonstration 
of skills and knowledge. LEP students can hear a long explanation in 
English and may not understand. They see something being done and 
they can mimic the movements. But the model will not be made into an 
acquired skill until the students can practice it, and produce the required 
process of active steps on their own. 

An increasing number of educators have put the name of "cognitive 
apprenticeship" on the optimum learning process that takes place when 
working on something like a well designed project. It is what vocational 
teachers have known for years: that a goal of the classroom should be to 
have students engage in an "authentic activity" as a routine matter that 
takes theory into practice. 

A well-designed project does that. Therefore, the vocational education 
classroom, with its accent on producing publicly visible and meaningful 
work, is the setting most well-positioned for situated learning projects to 
take place. 
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HOW? 



To design and impiement a good project involves much the same process 
as the more extensive portfolios, but with a clearer beginning and ending, 
and a shorter time in between. 

Like portfolios or any other "authentic activity" used for assessment, you 
must clearly establish two basic points before starting a project: 



• The specific learning goals on which the work will focus. Both 
the teacher and the student should be very clear about 
exactly what they hope to learn by doing the activity. That 
gives the focus needed to see which activities will contribute 
to your aim and what steps are needed to achieve it. 

Portfolios, Chapter 8, pages 9-22, gives more detailed 
information about this step. 



• How it will be determined if the goals have been reached. 
What are the criteria for deciding if the work done on the 
project is an "achievement" and to what degree? It is 
important that students be able to see their own progress and 
know exactly where they stand. 

^ Portfolios, Chapter 8, page 31-32. 
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Rgures &-1 and &-2 are examples of how the developers of the Career 
Technical Assessment Project (C-TAP) offer California high school students 
a format for organizing the projects that will go into their portfolios. 

Rgure 9-1 (pages 16-17) is the initial, more detailed process for the student 
to thoughtfully work through all that is involved. 
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C-TAP Project Plan Organizer 

Use this organizer to record information and ideas for your project plan, tf you are planning a group project each team member must 
have separate tasks for which he or she is responsible. Therefore, it may help rf each team member completes an organizer, focusing 
on his or her responsibilities. Your group plan can include each team member's organizer information. 

AimodhePfojMt 

lAve plan to accomplish: 



Doscnption oC Activitiee 

Break the activities into steps below. Use as many steps as you need. Estimate how long each step will take to complete (number 
of hours or days). 

Step 1 . 



TimeRequired:_ 



Step 2. 



Time Required:_ 



Step 3. 



Time Required: 



Step 4. 



Time Required:_ 



Step 5. 



Time Required: 



Step 6. 



Time Required:_ 



Step 7. 



TimeRequirod:_ 



Pago 16 



Rguro 9-1 

Source: Career-Technical Assessment Project Portfolio - pps. 11-12, Student Guidebook 
California Department of Education and Far West Laboratory 
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step 8. 



Time Required: 



Step 9. 



Time Required:. 



Step 10. 



Time ' '^uired: 



Add more sheets it necessary. 
Materials needed 



Who outside the 
classroom will be 
involved and how: 



Cvalualion 

Evidence of Progress; 



Final Product(3): 



When you have completed your organizer, use the information to write your Project Plan. Remember to follow the format used in 
the sample project plans. Be sure to include in your plan as many details as possible for each step of your project. 

Figure 9-1 (con't) 

Source: Career-Technical Assessment Project Portfolio - pps. 11-12, Student Guidebool< 

California Department of Education and Far West Laboratory Page 17 
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Rgure 9-2 (below) provides a concise synopsis sheet to give the student 
a quick overview reference point of the entire process. 



Project Plan Form 

AIM: 



DESCRIPTION: 



EVALUATION: 

Evidence of Progress (at least 3 types) 



Final Product(s) 



Figure 9-2 

Source Career-Technical Assessment Project Portfolio - pps. 11-12, Student Guidebook 
California Department of Education and Far West Laboratory 



ERLC 



Page 18 



Crosspaths Management Systems, Inc. 

255 



Projects ~ Chapter 9 



Figure 9-3 (pages 20-21) is another way that the process involved in 
projects can be put into an organized format for the student to follow. The 
format serves as the record form of projects done for portfolios at the 
Rindge School of Technical Arts, in Cambridge, Massachusetts. 
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Rindge School of Technical Arts 
Industries Portfolio Record Form 



Name: 



Industry Group: 



Teacher: 



Describe the project that you have been working on: 



Describe what is happening 
in the photograph 



Glue Picture Here 



What material, tools and/or 
resources did you use? 



Source: Rindge School of Technical Arts 
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How long did this project take? Describe all the steps involved in 
doing this project. 



If you worked in a team, list the names of your teammates and 
describe what you contributed to the completion of this project. 



What skills did you have to use or develop to carry out this project? 



How do you feel about what you accomplished? (In what ways are you 
satisfied with what you did? In what ways dissatisfied?) 



What are some ways you might apply what you learned in doing this 
project-either in other classes or in your life outside of school. 



Figure 9-3 (con't) 

Source: Rindgo School of Technical Arts 
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Important Points To Remember When Developing and 

Using A Project 



I. The Element of Choice 

For students to fully integrate the knowledge derived from their work, they 
must feel that it is theirs; that learning has been an active experience with 
interest and meaning for their own lives. When students make a choice from 
various options, or take an idea and "run with it" at their pace and to their 
application, the needed feelings of ownership and consequent responsibility 
are fostered. 

For example, see pages 7-12 of this Chapter 

We listed examples of projects offered as part of the 9th grade curriculum 
at Rindge School of Technical Arts, in Cambridge, Massachusetts. In each 
of the four integrated areas of study, the student was asked to pick a first 
and a second choice preference from among the various project 
possibilities. 

An earlier section of this Chapter also lists some of the ideas from which 
California students can choose the four Work Samples that will be included 
in their C-TAP portfolios. 

» See pages &-7 of this Chapter. 



^ Figure 9-4 (next page) shows a format for the students to keep track 
of their choices, in terms of what skills they demonstrate. The 
particular skill area shown here is Child Care Occupations. 
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Rgures 9-5 and 9-6 are samples of guidelines for a student to select "Work Samples" to 
include in the C-TAP portfolio as demonstrations of the targeted skills. In this particular 
example, the subject area is Home Economics, specifically the skills needed in Child Care 
Occupations, 

Rgure 9-5 (below) illustrates ideas given to the student as a basis for choosing which 
Work Samples they will develop into proof of competency. 



Ideas for Work Samples in Child Care Occti(>atkxis 

Below are ideas for work samples. In some cases, documentation {photographs for 
instance) would be included in the portfolio, rather than the product itself. 

♦ Flannel board created to tell a story to preschool children (target skills 1,2,3) 

♦ Lab Activity Plans (with objectives, materials, procedures, and observation notes) for five 
subject areas (target skills 1 , 3, 8) 

♦ Role play as "teacher* explaining work site safety procedures and rules/policies to "parents" 
and their very active "child" (target skills 5, 6, 7) 

♦ Recipes and poster showing children working together to make two healthy snacks (target 
skills a 3, 4) 

♦ Chart comparing the equipment and safety features of the outdoor play areas at three day 
care centers (target skills 2, 5) 



Source: Career-Technical Assessment Project Figure 9-5 

California Department of Education and Far West l_aboratory 

Figure 9-6 (below) is an example of a "caption" a student wrote to describe the Work 
Sample project that she had chosen to do. 

Example of a Caption to Accompany Picture of Work Sample 



Name: Christina Alvarez Date: January 17, 1992 

Title: Flannel Board Story: The Hare and the Tortoise 

My work sample includes the flannel pieces I made for the story "The Hare and the Tortoise," a 
photograph of my final story board, and a cassette tape (audio) of me telling the story and the 
children talking about it. I chose this work sample because it shows three of the required skills. 
I chose a story that 3- to 5-year olds will understand and created a "learning circle" where children 
can sit, see the flannel board, and discuss the story. I used positive guidance in telling the story, 
managing the children, and having them talk about the Hare (who thinks he will win because he 
is faster) and the Tortoise (who keeps on trying). 

The presentation went well and the children liked it and understood it. When I do this again, I 
might let the older children put up some of the pieces on the flannel board. 



Source: Career-Technical Assessment Project Figure 9-6 

California Dopartmont of Education and Far Wes\ Laboratory 
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11. The Element of Reality 



How will the work be connected to the outside world and put into a 
meaningful context? A purpose of a project is. for students to project what 
they have learned beyond textbooks and the classroom walls. How can 
skills be applied to real situations that add the experiential learning 
dimension which characterizes growth? 

^ Examples of projects are given on pages 2-12 of this Chapter. 

The students can choose their projects from a variety of options, but all of 
the work involved leads the students out of the classroom and into their 
community. 

111. The Element of Exhibition 

The process of a project is designed to actively gain determined skills. The 
product of a project is a display of the skills gained, the way the students 
will provide visible proof of what they have learned. How the students 
demonstrate their mastery of certain skills should be done through some 
type of exhibition planned for presenting to an audience. 

Preparing an exhibition calls for proof of the student's understanding and 
of some imaginative capability, making a project an assessment agent as 
well as an expressive tool. It also provides LEP students tangible channels 
of expressing their skills other than the written or spoken word. 

Having a wider audience than just the teacher or another student view the 
finished work is another way of connectirig the project to the outside world. 
Notice in .the example of the project developed at Bell Multicultural High 
Schoc! in Washington, D.C., the project was aimed not just at the immediate 
community, but for possible distribution across the country. 

83r See page 4 of this Chapter. 

A broad audience also can be an incentive needed for the students with 
weak writing skills to make repeated editing attempts. 
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isr Rgure 9-7 (next page) gives a format for students to follow when 
preparing to present their finished project work to an audience. It 
ensures that the student is organized, practiced, and equipped with 
audio/visual aids when doing an oral presentation before an 
audience. Those are essential guidelines that are especially 
important for LEP students. 
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C-TAP Project Presentation Organizer 



The Checklist below lifts steps to prepare an oral presentation about your C-TAP project. Record the date of your Project Presentation 
below. Fill in the dat> when each of the following steps should be completed. As you complete each step, fill in the completion date 
and check It off in the box to the right. 



Date of my Oral Presentation: 



Checklist 

Due Date 

Date Completed 

Plan what I want to say, including: 



How to introduce my project and what the 
audience will need to be told about it: 

What I did: 

Who helped me with the project: 

Any interesting, unexpected, difficult things 
that happened and how I handled them: 

What I think of my project now that its done: 

What I found out by doing the Project, and what 
I learned about myself: 



Think about possible questions the audience might have 
and how to answer them 

Plan visual aids to help the audience understand 



Make the visual aids 



Practice my presentation to make sure that: 



♦ I am familiar with what I want to say; 

♦ It is organized and makes sense; 

♦ It is between 5 and 10 minutes long; and 

♦ My visual aids are useful. 

Tell the teacher if I need audio/visual equipment 



I I Invite parent{s), other teacher(s), contact(s) 

I 1 outside of school 

Figure 9-7 

Source: Caroor-Tochnical Assessment Project. California Department of Education and Far West Laboratory Page 27 
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HOW Do Some Schools Foresee Projects Being Used 
Even More Extensively? 



A more ambitious type of project, called the Senior Project, has been 
successfully implemented within some high school academic curriculums. 
As a way to reach an integrated curriculum, vocational education programs 
are also beginning to try Senior Projects. In some cases, schools have 
made the Senior Project into a required credit for graduation. 



Basically, what is involved is students will choose a topic to explore and 
develop their senior year of high school. Ideally, at least in the eyes of the 
vocational education programs, it is a topic that also responds to a need 
in the community. The stages to follow with the chosen topic are: 

• Research (gathering of primary and secondary source 
information) 

• Analysis 

• Application 

• Production 



Schools can adapt the Senior Project to their own settings. For example, 
the Paul Hodgson Vocational-Technical School in Newark, Delaware, has 
had the "Senior Project: An Exhibition of Achievement" underway since 
1990. The staff designed the "Senior Project" as a process of bringing 
student-centered research to the development of a career related product 
and a final public presentation. The Project gives the guidelines for 
students in the vocational program to demonstrate their integrated 
academic knowledge through authentic performances. 
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At the Rindge School of Technical Arts (RSTA) in Cambridge, 
Massachusetts, the staff has been studying how the Hodgson School 
successfully implemented a Senior Project which was so encompassing that 
it meant major curriculum and schedule accommodations. RSTA is in the 
process of debating how they can "customize" Hodgson's experience to 
their own program needs, but basically the plan envisions: 

• Seniors attending a once/twice a week seminar, preferably co-led by 
an English teacher and a technical teacher. The student will receive 
either a Social Studies or an English credit for the project. 

• Each senior will have an advisor, mainly drawn from RSTA technical 
teachers, with whom the senior meets once or twice a week. 

• Each senior will have a community/industry mentor with whom s/he 
meets periodically and presents progress reports to. 

• Before actually implementing this as part of the curriculum, RSTA is 
aware that thought will have to be given to addressing the language 
credit component for LEP students. Demonstrated proficiency in a 
language, even other than English, has been accepted as a valid 
language credit by the Cambridge school system. 

The community involvement is essential to RSTA's concept of making a 
project into scholastic credits. Having a community connection to the real 
world is seen as raising the standards of work done by the student, and 
also makes the vital connection for integrating school and the real world. 
The students choose their projects, but ideally, the school should have 
them defined by the community's needs. 

A good example of how RSTA hopes to accomplish this, is the very project 
that will be starting off this "experiment". 

In an outreach recruitment activity at another school's 8th grade, the team 
from RSTA was told about the school's need for stage lighting so the 
students could do good theater productions. RSTA seniors concentrating 
on electronics in the vocational section will work with seniors in English and 
theater classes in the "academic" section of the school to research theater 
productions, the writing and directing of plays, and then design, build, and 
install stage lights and equipment for the lower school. 
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WHERE? 



Points of Contact That Have Developed and Used 
Projects 



Bell Multicultural High School 
3145 Hiatt Place, N.W. 
Washington, D.C. 20010 
(202) 673-3551 

Contact: Ana Vasquez or Kevin Farley 

Brevard Community College 
250 Grassland Rd., S.E. 
Palm Bay, Florida 32909 
(407) 951-1060 
Contact: Kathy Tailey or 
Sue Lundgren 

C-TAP/Far West Regional 

Laboratory 

730 Harrison Street 

San Francisco, California 94107 

(415) 565-3000 



Hodgson Vocational-Technical 
High School 

2575 Summit Bridge Rd. 
Newark, Delaware 19702 
(302) 834-0990 

Michigan State Board of 
Education 
P.O. Box 30008 
Lansing, Michigan 48909 

Rindge School of Technical 
Arts 

459 Broadway 
Cambridge, MA 02138 
(617) 349-6753 



268 



Page 30 



Crosspaths Managoment Systems. Inc. 



Projects - Chapter 9 











Written Sources of Information on Projects 



Crandail, Jo Ann, "Directions in Vocational Education for LEP Students and 
Adults", (1985). Columbus, Ohio: National Center for Research In Vocational 
Education. 

Duran, R.P. "Testing of Linguistic Minorities" (1989). In R.L Lynn (ed.). 
Educational Measurement (3rd Ed., pp. 573-587). New York: American 
Council on Education. Macmilian, 

Fradd, Sandra H. and Weismantel, M.J., Meeting the Needs of Culturally 
and Linguistically Different Students: A Handbook for Educators. (1989). 
Boston, Massachusetts: Little, Brown, and Company. 

Grognet, Allene, "Work Skills and The Limited English Proficient Student", 
(1990). Washington, D.C.: Center for Applied Linguistics. 

Hamayan, E., and Camico, J., (eds.). Limiting Bias in the Assessment of 
Bilingual Students. (1991). Austin, Texas: Pro-Ed. 

Hamayan, Kwiat, & Perman, Assessment of Language Minority Students: A 
Handbook for Education . (1 985). Arlington Heights, Illinois: Illinois Resource 
Center. 

Little, J.W., Stretching the Subject: The Subject Organization of High 
Schools and the Transformation of Work Education . (1992). Berkeley, CA: 
National Center for Research in Vocational Education. 

Raizen, Senta, R eforming Education for Work: A Cognitive Science 
Perspective. (1989). Berkeley, California: National Center for Research on 
Vocational Education. 

Resnick, Lauren, "Assessing The Thinking Curriculum: New Tools for 
Education Reform", (1992). In Gifford (ed.), Changing Assessment: 
Alternative Views of Aptitude. Achievement and Instruction. 
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Shepard, Lorie, "Commentary: What Policy Makers Who Mandate Tests 
Should Know About the New Psychology of Intellectual Ability and 
Learning", (1992). In Gifford (ed.), Changing Assessments: Alternative 
Views of Aptitude, Achievement, and Instruction. Boston: Kluwer Academic 
Publishers. 

Stasz, C, McArthur, D., Lewis, M., and Ramesy, K., Teaching and Learning 
Generic Skills for the Workplace. (1990). Santa Monica, California: The Rand 
Corporation. 

Stiggins, Richard J., "Revitalizing Classroom Assessment: The Highest 
Instructional Priority", (1988). Phi Delta Kappan. Vol. 69, No. 5. 

Pierce, LV., and O'Malley, J. M., "Performance and Portfolio Assessment for 
Language Minority Students", Program Information Guide, Series, 9. (1992). 
Washington, D.C.: Department of Education, Office of Bilingual Education 
and Minority Language Affairs. 

Zessoules and Gardner, H., "Authentic Assessment: Beyond the Buzzword 
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Standardized Tests 



WHAT? 



It seems almost superfluous to explain what something is when that 
"something" has been a part of most people's lives since kindergarten. Just 

say "This is a standardized 
test" and people immediately 
see sealed booklets being 
passed down the row and 
number 2 pencils being 
sharpened. Many might 
also feel sharp pains in their 
stomachs and cold sweat on 
their faces since stan- 
dardized tests are often 
synonymous with high 
stakes, with results that have 
enormous impact on our 
lives. They are frequently a 
determining factor in 
deciding which school we 
can go to, what class we will be in, what job we will get,., in short, which 
direction our lives will take. That makes standardized tests a pervasive and 
powerful part of our educational system. 
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As a big part of our lives, standardized tests are also a big business. 
Statistics from studies done on the administering of standardized tests vary 
slightly, but it is estimated that: 



A total of 20 million school days, at a rough cost of $700 to 
$900 million, are spent on giving the 127 million separate tests 
that American school children take annually as part of the 
standardized batteries mandated by states and districts, 

♦ National Ccmmisston on Touting and Public Policy, 1990» p.8. 



And the monies generated by standardized tests have increased, even 
when student enrollment has declined. 



Growth in Revenues 1960-1989 

In Test Sales and Public School Enrollments 
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To put such staggering figures into a more manageable context closer to 
home, we can take a look at the example of one fairly typical large school 
district. The following facts were gathered by the Office of Technology 
Assessment, U.S. Congress, from the director of Testing and Evaluation in 
an urban public school district for the year 1990-1991: 



Total Enrollment: 
Teachers Employed: 
Total Budget 
Tests Administered: 



191.000 students (140,000 of which are tested) 
12,000 (including special education personnel) 
$1.2 billion 

Norm-referenced questions supplemented with locally 
developed criterion-referenced checklists; purchased 
from large commercial publishing company that machine 
scores the tests and provides computer-generated score 
reports to district personnel. 



When Administefed: Twice a year (fall and spring) 

Annual Materials' Cost of Administering: 

$1 .6 million ($ 5.70 per student per test administration). 

Salary Costs for Teacher Time Spent Administering Tests: 

$3.6 million ($ 1.8 million per testing 
cycle). 

Salary Costs for Teacher Time Spent in Classroom Preparation of Students for 
Standardized Tests (varied with teachei): 

Case A= 1.5 weeks of preparation: $ 7.2 million 
per administration 

Case B= 3 weeks of preparation: $ 15.3 million 
per administration 

Total Salary Costs for Teacher Time (Administratkxi + Preparation): 

Case A= Per session: $ 9.0 million 

Per Year: $ 18.0 million 



Case B= 



Per session: 
Per Year: 



Total Testing Costs (Materials + Teacher Time) To District : 



Case A= 



Case B= 



Per session: 
Per Yean 

Per session 
Per Yean 



$ 15.3 million 
$ 30.6 million 



$ 10.6 million 
$ 19.6 million 

$ 16.9 million 
$ 3Z2 million 



Source: Office of Technology Aaaessment, 1992, pp 26-27 
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Standardized tests are clearly high stakes and high cost tools in our 
educational structures. But technically, in the basic nuts-and-bolts 
functioning sense, what are they? 



U.S. Department of Education's Official Definition 

A test is standardized if ft is based on a systematic sampling of behavior, 
has data on refiability and validity, is administered and scored according to 
specific ffistructions, arid is widely used. A standardized test may be norm- 
referenced or criterion based. The tests may, but need not, relate to 
readability levels, grade level equivalencies, or competency teased 
measurements, ( Federal Register, August 8, 1989, p. 34435,). 



To make life simpler and our baseline definition clearer, let's break down 
that official, technical version bit by bit. 



"a systematic sampling of behavior" 



A certain part of the 
population has been grouped 
and studied under structured 
conditions. Their responses 
to developed stimuli (questions) have been recorded and ranked using 
statistical analysis. 



"reliability" 



This tells you if you can trust the scores of a test to be 
repeatable or stable, able to be duplicated under the 
same conditions ana not registered merely as a fluke. 
When a test is being developed, the results that are 
found to reoccur during the trial testing stage are calculated with a 
statistical formula to get a "reliability coefficient". A coefficient of at least 
+0.80 or higher is usually considered necessary for a test to be "reliable". 
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Refers to whether or not a test actually measures what it 
pretends to measure, and nothing else. In theory, you 
can make inferences about a person's possible 
performance based on test scores. " Validity" is whether 
those inferences gathered from the scores recorded when performing a 
determined task are valid or not. 




•Specific instructions" 



These are the "structured conditions'' 
mentioned under "systematic sampling of 
behavior". Factors such as time, allowance 
for repetition or elaboration of instructions, 
guidelines for accepting or rejecting responses, and stopping the test are 
the usual steps that must be taken in accordance with a specific test's 
instructions. 



Yiormnreferenced" 



These types of tests give scores that can be 
compared to how a studied group normally 
performed on the tests. Using the "norm" score 
as a reference point, you can see if the test 
taker performed above the average or below the average of what other 
people have usually done on the same test. This is similar to the "grading 
on the curve" concept, seeing who is doing the worst and the best in class, 
depending on how everyone else has done. 



••criterion-based" 



These types of tests measure a person s 
performance in reference to established, carefully 
defined standards (such as 90 points on a test 
means getting an A), rather than to other people. 
Items on the test are chosen on the basis of their importance, not how well 
they differentiate among people. 



"grade level equivalencies" 



These are the tests wh'ch give scores 
that have been normed on children in 

the elementaiy, middle and secondary 

school grades. For example, a score of 
4.8 means that the test taker is functioning at a level typical for a fourth 
grader in the 8th month of the school year. 



CroGspothft Managomont Systems, Inc. 



Page 5 




275 



Standardized Tests - Chapter 10 



•Readability levels" 



Reading inventory comprehension tests 
commonly do percentage scoring of answers to 
give 3 general levels of abilities. Percentages 
are not exact because some flexibility should be 



given on both the high and low sides: 



1) Independent Level (90%-100% correct) = the student can 
understand the text involved independently with no need of 
instruction. 

2) Instructional Level (70%-88% correct) = the student will understand 
the text involved with normal teacher assistance. 

3) Frustration Level (50% or below) The text involved is too difficult for 
the student to attempt with any chance of success or meaning. 

Other ways for registering readability levels are by degrees of proficiency 
established by certain programs or states, and as grade level equivalencies. 



"competency-based measurements" 



Tests that give scores in 
terms of the degree of 
accomplishment of stated 
objectives. 



To pull it all back together again, we can therefore define a 
"standardized test" as one that is: 

• developed from statistical studies of responses from 
certain groups of people; 

• administered under standard, pre-set conditions; and 

• designed to measure a specified skill. 
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WHY? 




WHY Are Standardized Tests Used? 



Formal testing in schools with standardized procedures started its official 
way into the Western world at the turn of the century with the work of Alfred 
Binet and his French colleagues. Binet was asked by the educational 
leaders of Paris to help determine which students would succeed, and 
which would likely fail in the elementary schools. His solution was to 
develop a large set of items that would prove the most discriminating 
among young children in light of his particular goal. This work ultimately led 
to the first intelligence tests and the calculation of an IQ, or intelligence 
quotient. 

In the U.S., World War II brought the real boom to the commercial, 
stcidardized testing for *trainability" purposes. The Armed Forces had to 
deal with floods of new recruits to be trained during the war, and schools 
had to process the thousands of applicants using the G.I. Bill to continue 
their education after the war. 



Having the Army lead the way in implementing, expanding, and legitimizing 
mass tests in this country helps to explain why a pen/asive standardized 
testing mentality has developed. In education, that mentality is reflected as 
a "uniform view of schooling": students encounter identical curriculum, 
subjects are taught with same basic procedures, and students are 
evaluated according to the same formal, "standard" instruments. 
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There are some cases and times when a standard view is needed. 
Standardized tests in themselves are not always bad; it is the way that they 
are used and how their results are used that can produce negative effects. 
When we look at why they are used heavily throughout education, we see 
major reasons that could be interpreted positively or negatively. 



WHY Are Standardized Tests Considered Useful 
Assessment Tools In Some Situations? 



1) Official Accountability 

Tests are increasingly a legal requirement for local, state, and federal 
educational programs. According to a study on testing in American 
schools, 46 states had mandated testing in 1990, as compared to only 29 
in 1980, (OTA, 1992, p,2), 

2) Comparison of Programs 

Administrators and policy makers often gauge a program's effectiveness 
through the results of mass testing, especially in tight budget times. In fact, 
"Outcome-based measures of the effectiveness of educational programs - 
generally achievement test scores- have become key elements in the 
congressional appropriations and authorization process," (OTA, 1 992, p, 
2). 

3) Comparison of People 

When you apply for a job or to college, tests compare the preparation 
levels of the applicants. Tests are used to show who has made the most 
progress, who still has needs to be met, and even who is the most 
"suitable" person to date, 

4) Instructional Decision Making 

Publicizing low national test scores served as a large impetus for the 
movement towards a more "competitive" curriculum and efforts to 
implement education reforms throughout the last decade. 
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WHY Standardized Tests Should Be Used Very 
Cautiously. Especially With LEP Students 





We started with the "good news" possibilities of standardized tests. The 
controversies raised by the results of even some of the best intentions of 
the tests and their users make the "bad news" side one that carries a 
tremendous amount of weight and cause for concern. Why? 



1) There are Tests That Don't Live Up to Name or Claims 

By definition, all standardized tests are based on a theory, but the theory 
isn't necessarily based on fact. There are tests which are inaccurate in their 
stated purpose or particular name. 

This malady can happen even to the "sacred cows" among standardized 
tests, like the SAT. For decades, the Scholastic Aptitude Test has been one 
of the definitive inputs in the decision about a student's college potential. 
Each year, 1.5 million students have implications about their supposedly 
raw intelligence- or lack thereof- determined by their performance on the 
SATs. 

Yet, last year the publisher. The College Board, recognized that the test 
actually measures achievement as well as developed ability, and cannot be 
accurately described as a test of "aptitude". 

Some of the proposals for the needed new name give a sardonic summary 
of many peoples' frustrations with what they viewed as a biased test. Since 
the College Board wanted to keep the acronym of SAT, tongue in cheek 
suggestions ranged from Students Are Tricked, to Sexist And Trivial. 
According to the test's critics, any of the more serious suggestions, such 
as Student Assessments for Transition, or Student Attainment Test, would 
just be cosmetic touches to mask alleged bias problems with the test. 

The final decision by the publisher was to change the test's name to 
Student Assessment Test. 
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If we look at another example of a test not really testing what it says it does, 
we see that even the best of intentions can invalidate results. 

One of the most commonly administered tests of children's abilities was 
carefully translated into Spanish appropriate for Mexican-American students. 
The part of the test that was supposed to test short-term memory seemed 
simple enough-- just translate the sequence of numbers the children were 
expected to repeat back. 

Yet, when the numbers were translated into Spanish, the English, single- 
digit quantities turned into twice as many syllables. And since Spanish has 
fewer vowel sounds and consonants than English, the translated words 
sounded more similar to each other than the English number words did. 

Both of these effects, acoustic similarity and length of stimulus items, have 
been shown to significantly affect memory tasks. Field testing the 
translation proved that the Spanish test group did indeed score lower 
(O'Conner, 1989). 

Translating the words of the test also translated the purposes of the test. 
It no longer was a fair assessment of a child's short-term memory. 



2) Tests Can Produce "Setf-Fuffilling Prophesies" 

Research has repeatedly shown that teachers' expectations of students, 
consciously or unconsciously, are affected by knowledge of test scores. 
As early as 1909, this occurrence was studied as the "Looking Glass 
Theory", the tenet that people tend to adapt behavior and self-perceptions 
to how they see themselves as viewed by other people. {Lewis, 1987). 



Good test scores can make teachers view students positively. 
Consequently, teachers create high expectations which they encourage the 
students to fulfill. On the other hand, low test scores can produce low 
expectations and little encouragement. 
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A Sample Study 



A classic study of this phenomenon created quite a stir when it was 
published in the 1960s, and it is still frequently referenced and debated 
today by many in the field of testing. 

A research team from Harvard University administered a well-known 
standardized test of general abilities to the students in a San Francisco 
elementary school. The teachers were told, however, that the "Harvard Test 
of Inflected Acquisition" was being used, a false, but purposely imposing 
name for the test. The teachers were also given falsified results, not 
knowing that experimental and control groups had been established for 
research purposes. 

Although the labeling was done by experimental and control grouping, and 
not by the test results at all, the teachers were told that 20% of the students 
were "late bloomers ready to bloom"; the students who would show 
"potential spurts" in intellectual growth during the ensuing year. 

At the end of the year, the students were tested again and teachers' 
opinions surveyed. The children in the experimental group who had been 
labeled as "potential spurts", regardless of their initial test scores, registered 
the greater gains in the standardized test's scores and in grades in reading. 
They were also the students rated by teachers as being "happier, more 
intellectually curious, and less in need of social approval." A demographic 
breakdown of the results showed further that the greatest gains within the 
experimental group occurred among Mexican-American boys. 

In other words, teachers' positive expectations, fostered by tests scores, 
seemed, in turn, to actually raise tests' scores and grades. 

The interpretation of these results was the basis for the title of the book. 
The Pygmalion Effect , that reported the study. Drawing from the George 
Bernard Shaw play about a professor transforming a "low-class" flower girl 
into a "fair" lady, the preface of the book quotes the play's main character 
as saying: 



The difference between a lady and a flower girl is not how she 
behaves, but how she is treated." 

- Eliza Doolittle, Pvqmalion 
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3) Tests Can Displace Focus of Instruction 

Bringing standardized tests into the classroom can take assessment away 
from the integral parts of learning and the main characters, ie., what is 
happening on a day to day basis between the teacher and the students. 
Tests can create the instructional time "detour" of teaching to the tests, with 
the consequent loss of real productivity and gains. Studies have quantified 
the loss as follows: 



• It takes an average of 140 hours of intensely focused 
instruction to produce ONE grade level rise on standardized 
test (Thomas et. al., 1991). Is that what you want so much of 
your teaching time strictly focused on, trying to raise by one 
year what is a dubious concept anyway,especia!ly for older 
students? 

• For LEP students, the time loss is even more pronounced. 
Studies done by linguists on how much time is needed for LEP 
students to reach the 50th percentile ranking on standardized 
tests yielded the following averages: 

If an LEP student was under 12 years old when he/she 
arrived in the U.S. and had at least 2 years of schooling 
in their native language, 5-7 years are needed. 

If the student arrived in the U.S. between the ages 4 
and 6, with little or no schooling in native language, 7-10 
years are needed. 

For those that arrive in the U.S. during adolescence, 
7-10 years are needed. 

- Collier, 1987 

In other words, if standardized tests are used as primary indicators, 
for 5 to 10 years in the lives of LEP students they will be considered 
well below average in many skill areas just because they don't have 
the standard language or background that the tests results are 
measured by. 
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4) Results Have Been Used to Mislabel and Misplace Students 

The majority of standardized tests are normed on native English speakers, 
making results non-applicable when language interference factors are not 
taken into account. 



That was the premise validated in a lawsuit brought by Asian American 
families in Philadelphia. In response to the lawsuit, an elementary school 
evaluated 50 LEP students through an observation process :ather than 
standardized tests. Under the test-based decision process used in the 
past, most of the 50 children would have been placed in special education. 
Under the observation system, only four were. (Fair Test, 1990, p. 9). 



On the national level, it is estimated that 40% to 50% of LEP students have 
been mislabeled based on results scored on standardized tests. (Sosa, 1 990). 

Why are so many mislabelings, misinterpretations, and misplacements 
made when LEP students are judged on the basis of traditional 
psvchometric tools? If we look at the different linguistic/cultural dimensions 
that come into play and place them against the "norm" that is expected, we 
get a clearer picture of what is involved. 



Cultural Behaviors Which May Impact Test Performance Resufts 



The fifteen factors we explain here are by no means the only cultural 
behaviors with potential impact on performance results, nor should they be 
considered as universals in any culture. Culture is learned at a local, 
personal level, so there will always be differences, even among people in 
a similar group. 



With the reminder that ALL students are different and unique, we can 
examine some of the variables that make a difference. 
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1. Movement 

Bodily movements differ across cultures both in their types and range. For 
example, in some cultures, people typically "talk with their hands", being 
more actively expressive when they speak. 

In many U.S. classrooms, however, a passive style is rewarded, and 
students whose style is more active are often referred for behavior 
problems. A standardized test situation is almost always a passive one, 
sitting quietly for a long period of time to follow imposed instruction. 
A person with a more active, independent style might perform poorly in 
such a setting. 



II. Space 

People from different cultures use, value and share space differently. 
For some persons, close proximity to another while talking is an invasion 
of private space; for others, it is a natural placement showing support and 
attention. 

In the typical U.S. assessment process, taking a test is a formal procedure 
requiring certain distance between the test administrator and the test taker. 
For students who value proximity, the physical distance is seen as an 
emotional gap that can make the testing situation stressful and un- 
comfortable, and performance could consequently suffer. 



111. Time 

In the United States, "time is money"; speed and efficiency are highly valued 
goals. In contrast, other cultures see time as more expendable, and 
interpersonal exchanges take priority over punctuality. 



Most standardized tests are "speed tests", based on the assumption that 
all students have learned to move through tasks as quickly as possible. 
Since scores are tied to time, cultural variations of the concept directly 
affect the results. 
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IV. Interactions 

Another cultural variable in the concept of time is the orientation towards 
monochronic or polychronic activity. In a monochronic culture, people are 
encouraged to do one thing at a time; in polychronic cultures, people are 
generally used to handling various interactions at the same time. 



The maxim "business before pleasure" reflects a monochronic orientation 
and the preferred approach in the U.S.. Assessment is usually operated on 
a monochronic basis, and tests are structured for a student to methodically 
take one step at a time within a specified time frame. These monochronic 
features inherent in a testing situation can create extraneous difficulty for a 
student accustomed to a polychronic approach. 



V. Bio-rhythm Adaptation 

A culture's scheduling of time can set a person's internal bio-rhythm time 
clock. Ben Franklin captured the American valued preference for schedule 
with his maxim: "Early to bed, early to rise,/ Makes a man, healthy, wealthy 
and wise." 



Yet, there are some countries where offices close up for several hours in 
mid-afternoon and don't finish business until 8 PM; restaurants don't even 
open for dinner until 9 PM; and the theater starts at 10:30 P.M. Evenings 
are a time of activity and the morning something to be leisurely started. For 
a student raised in this culture, it is disorienting to deal with the American 
preference for getting "students early while they are fresh," a mentality that 
usually schedules tests for first thing in the morning. 



VI. Cognitive Style 

Increasingly, the difference in the continuum of cognitive styles is being 
made synonymous with the different functioning between the left and the 
right hemispheres of the brain. 
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The differences of the two with the corresponding approaches to 
assessment have already been noted in 

ra* Chapter 4, Assessment, pages 2-3. 

Here, therefore, we merely note again that the traditional testing, 
assessment situation favors the left-brain methodical, reflective, rational 
approach over the more intuitive, global, and hands-on, right-brain style. 

VII. Perceptual Style 

The environment in which a person is raised affects the perception of the 
world and its objects. For example, present a picture of a horse to 
students from different backgrounds, and you can get different views of the 
same animal. A person raised in a c.ilture where horses are used for work 
and transportation will focus on the horse's feet or mouth, A person raised 
in an urban setting might only know a horse through rides on a carousel 
and therefore the color is the important part of the animal. 



Research suggests that perceptual styles vary in the dimension of field 
independence/field dependence. A person with a field-independent 
perceptual style can pick up on specific details within a complex pattern, 
able to see them apart from a whole, in contrast, field-dependence 
perception means that you need the whole to be able to identify the small 
parts. 



in general, the formal testing process favors the field-independent 
perceptual style. Students who are able to perceive details will usually 
perform better on standardized objective tests than someone who has been 
raised in a culture that has cultivated global perceptions. 



VIIL Goal/Structure 



Most games played in the U.S. have a winner and a loser, and students 
learn early in life to compete in order to win. Doing better than others is 
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often proof of mastery, and American classrooms have traditionally favored 
such a competitive belief. 

In contrast, there are cultures that create cooperative societies in which 
working together is a mutual goal. Students raised in such cultures are at 
a great disadvantage in the typical testing situation that calls on individuals 
to keep their answers to themselves while they work silently and on their 
own, and to do their best, meaning better than others. 

IX. Gender/Role 

The effect that gender can have on assessment results has already been 
discussed earlier in this Handbook. 

■sr Chapter 7- Rating Scales, page 20, 

It should be further noted that gender-determined roles are in flux in many 
areas of the world, and two students from the same culture might have 
diametrical points of view or behavior in this respect. 

X. Role 

As an extension of the traditional emphasis on competition, individual 
achievement is sufficient motivation for most American students to do their 
"best". For a person who has been raised in a culture where their role as 
a family member is just as valued as their role as an individual, motivation 
will be tied more to family expectations than their own individual goals, 

XI. Locus of Control 

"You can do anything that you really want to do" extends the individual 
approach to feeling that a person is in control of their own fate. In the U.S., 
a great emphasis is placed on the belief in individual responsibilities and 
rights. 
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In contrast, there are other cultures that believe that control lies outside the 
individual; that forces, such as fate or spiritual beliefs govern the 
outcome of one's efforts. For a student raised in a '*fate" or spiritual system, 
he/she can trust a prayer before taking a test just as much as anything else 
done in preparation for getting good results. 



XIL Language Use Pattems 

The manner in which a person uses a language, knowing when different 
styles are appropriate and with whom lo use them, are developed language 
use patterns. LEP students can often "misread" a situation, and not realize 
that a particular language pattern or tonal register is being called fo*'. This 
could result, for example, in using informal English without realizing that a 
testing situation calls for more formal expression. 



Consequently it is not sufficient to draw conclusions about students' 
language proficiency based on just one icest or task. 



XIII- Language Loss 

The rate that LEP students lose their first language in the process of 
acquiring a new one might be greater at times than the gains being made 
in English. Loss of proficiency in a first language while learning a second 
one can create confusions that will interfere with performance on tasks 
directed to monolingual students. 



XIV. Code Switching 

Bilingual students often switch back and forth between languages, even in 
the middle of a sentence. Once, this was viewed as a weakness, as having 
to fill in words unknown in the other language. However, research has 
shown that this "code switching" is predictable and regular, since the rules 
of both languages are maintained throughout the exchange. 

Yet, if a student switches codes during an answer to a standardized test, 
usually the response has to be counted wrong, even if the content was 
right. 
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XV. Language Variance 

The communication of information is affected not only by different 
languages, but also by the differences possible within the same language. 
Just as a person from Birmingham, England, might be puzzled by the way 
a person from Birmingham, Alabama, speaks, so it happens when other 
people speak supposedly the same language. The Spanish that has 
evolved in Latin American countries has taken unique turns away from the 
original "Spanish from Spain" starting point, and the dialects among Arabic 
countries can be mutually incomprehensible. 



if a student is from a linguistic situation different from the evaluator's, or if 
the test was designed and normed on a population speaking a different 
dialect than the student being tested, then the student may be wrongly 
judged. 



Bsr Figure 10-1 on page 24 of this chapter will give you a continuum on 
which you can rate your students' acculturation level; their point on 
the scale of these cultural/linguistic dimensions that affect 
testing/learning perfonnance in an American classroom. 
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HOW? 



"Interpreting test scores calls for experience, flexibility, and 
willingness to remember that one's statements are probabilities 
rather than certainties. To expect more of a test is, at best, foolish. 
To claim more is at the least, naive," 

- Spolsky 

cited in DeAvila, 1990, p. 222. 



We have already pointed out why many grains of salt should be taken with 
each standardized test. Many educators complain that there are so many 
problems with the tests, why must their students take them at all? 

The What section of this chapter addressed that question by showing how 
standardized tests are a big part of the schooling process. They will 
continue to be so for quite some time. Since legal mandates require that 
they be administered, how can the process be done, the tests given and 
taken in the least harmful way possible? Within the constraints inherent in 
the very fact that the test has been standardized, there are two checklists 
that educators can note to better use and understand standardized tests. 
The checklists give the guidelines to find out: 

^ HOW to know if a standardized *est is right for you and your 
students 

^ HOW to make the most of a standardized test 
This section will also show: 

^ HOW some standardized tests are being used by vocational 
evaluators 
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HOW To Know !f a Standardized Test Is Right For 
You and Your Students 



Read the manuals and descriptive materials that come with the tests. Use 
them as a guide to help you answer the questions that shoulc". be 
addressed before using an assessment instrument. 



1) How have the test developers defined the skills and concepts they 
are proposing to examine with their product? Do they match your 
interpretation of the concept being targeted? The problems that 
can be created by a mismatch here are illustrated in 

^ Page 9 of this Chapter. 

It explains the misconception of what "aptitude" really means and 
how it is often misused by standardized tests. 



2) Knowing what the test is designed to do can help you decide what 
you want to do with it. Why are you using this instrument? What do 
you want it to accomplish? Too often, educators are given a test to 
administer, and even use it frequently, but don't know what exactly 
it is testing. 

For example, in 1991, a survey was taken in Texas among those 
responsible for carrying out vocational assessment under the Perkins 
Act. One hundred thirty professionals in the field followed the 
survey's format for listing the commercial instruments they used in 
the given categories of assessment areas, such as interests 
inventories, aptitude tests, and work samples. 



In the summary of conclusions to be drawn from the results of the 
survey, one of the five points noted was: "Those who completed the 
survey often listed instruments in incorrect categories or with 
incorrect names. There appears to be a great deal of uncertainty 
about what the instruments listed actually are designed to measure." 
(Kapes et.al., 1991, p.7). 
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3) For whom was this instrument originally developed and why? Who 
has taken this test and what are their characteristics? Are they at all 
similar to your students? 



4) What is the language background, ethnicity, and gender of the 
person(s) who developed the instrument? 



5) What is the validity of the instrument? (Remember the technical 
definition of validity is that a test actually measures what it says it 
measures.) The manual will give statistical results about the validity 
studies done when developing the instrument. 



6) How reliable is the instrument in being able to give consistent results 
over time? Again, the manual, if the test is expected to be taken 
seriously, will give the statistical results from reliability studies done 
when developing the instrument. 



7) What are the conditions for administering the test? Does it allow for 
interruptions so clarifications can be asked for and given along the 
way? 



8) How are the results communicated to the test takers? Are there 
instructions for giving feedback or doing a follow-up? 



9) Is the instrument culturally biased? If so, how? Can it accommodate 
varying value systems? 

When it comes to taking into account different value systems, you 
have to consider how many of the factors that influence a student's 
performance have a cultural/linguistic basis. 
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Figure 10-1 (next page) shows a Continuum of the CufturaJ and 
Linguistic Dimensions that you can use to rate your students in these 
important areas. It can help you determine where a particular 
student fits within the acculturation process needed to overcome 
problems with American standardized tests. 



The right side of the continuum are the factors that are most valued 
in the U.S. and reflected in the concepts and procedures of taking 
tests in this country. 



Therefore, the higher the score, the closer the match 
between a student and a traditional psychometric 
approach or test. TTie lower the score, the lower the 
level of acculturation and the greater need for non- 
traditional assessment approaches- 



Notice in the factors listed for the Continuum, only four of the dimensions 
of the learning/performing process are specifically linguistically linked. No 
wonder many in the field feel that "language proficiency" is too narrow and 
simplistic in classifying comn unication levels. The term "communicative 
competence" is thought by many to be a more accurate capturing of all that 
is involved. 



Pages 13-19 of this Chapter give explanations of the different 
dimensions and their impact on the testing/learning process. 
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A Continuum of CXiltural & Linguistic Dimensions 



II 



III 



IV 



V 



VI 



VII 



VIII 



IX 



X 



XI 



XII 



XIII 



XIV 



XV 



Active 



Close 



No strict Time 
Schedules 



Polychronic 



P.M. 



Intuitive 



Field-Development 



Cooperative 



Inequality 



Group 



External 



Mismatch 



Extensive 



Frequent 



-Movement- 



Passive 



-Space- 



Distant 



-Time- 



Adherence 
To strict Time Schedules 



-Interact ions- 



Monochronic 



-Bio-rhythm Adaptation- 
Cognitive Style 



A.M, 



Reflective 



-Perceptual Style- 
-Goal/ Structures — 
Gender /Role 



Field-Independent 



Competitive 



Equality 



-Role- 



Individual 



-Locus of Control- 



Internal 



-Langxiage Patterns- 
Language Loss 



Match 



Minimal 



-Code Switching- 



Infrequent 



-Language Variance- 



Nonstandard Standard 
Adapted fromtTraclical Consideration for Assessment of LEP Students. . Chamberlain and Medinos-Landurardi; 1991 
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1) Read the manual ! Check the points already highlighted in 



^ Pages 21 -24 of this chapter, "How to Know tf a Standardized Test Is 
Right For You and Your Students." 



2) Use the appropriate level. This can be a real problem for older 
students especially; also for those who have a low level of language 
skills, but a level of maturity that would be offended by, or find 
irrelevant, a low level test. 



3) Let learners adjust to the situation before taking the test; provide 
anxiety reduction experiences. 



4) Accept that even the most "comprehensive", "in-depth", "diagnostic" 
standardized test will give only a small part of a student's learning 
profile, and that small part is only in a limited situation. 



5) Cultural variables can also be reduced if test-taking strategies are 
taught. This shouldn't be interpreted as a call to 'teach to the test", 
but rather a recognition that the playing field should be leveled for 
all of those who are assigned to it. 
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HOW Some of the Standardized Tests Are Being Used 
By Vocational Evaluators 



To give an idea of how standardized tests are currently being used in 
vocational assessment, we present here the results of three different 
surveys. 

^ Rgure 10-2 (next page) shows the tabulated final results when 
professionals in the state of Texas were surveyed. The purpose was 
to investigate the actual use of vocational assessment instruments 
with students with special needs as part of meeting the requirements 
of the Perkins Act. It lists the most commonly used instruments with 
the respondents* opinion about each one. 
(Source: Kapes etal., 1991) 
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Resufts of Texas Survey of User Opinions of Assessment Instalments 

Assessment Instruments That Are Most Used: 



APTICOM 

Career Ability Placement Survey (CAPS) 

Career Occupational Preference System (COPS) 

Differential Aptitude Test (DAT) 

McCan^on-Dial Evaluation System (McC-DlAL) 

Occupational Aptitude Survey and Interest Schedule (OASIS) 

Pictorial Inventory of Careers (PIC) 

Becker Reading-Free Vocational Interest Inventory (RR/H) 

Social & Prevocational Information Battery (SPIB) 

Talent Assessment Program System (TAPS) 

Vocational Training Inventory & Exploration (VOC-TIES) 

Wide Range Interest Opinion Test (WRIOT) 



5-Point Ratinq 
Instrument 1 2 


3 


4 


5 


APTlCON'i 


1 1 


2 


10 


6 


CAPS 


" 2 


12 


6 


3 


COPS 


- 4 


2 


7 


3 


DAT 


- 3 


4 


7 


4 


McC-DIAL 


- 2 


2 


7 


11 


OASIS 


1 22 


15 


16 


1 


PIC 


- 2 


4 


6 


1 


RFVU 


2 3 


10 


12 


2 


SPIB 


- 2 


2 


1 


1 


TAPS 


1 - 


1 


11 


1 


VOC-TIES 




3 




5 


WRIOT 


1 1 


5 


5 





opinions 

6 7 8 9 10 11 1213 14 15 16 



- 3 5 1 

- 2 - - 

- - 1 - 

- - 1 - 

- - 3 - 
1 8 - - 

- 2 - - 

- 6 . - 



- 1 3 1 - 

- - 2 - - 

- - 1 - 2 

- - 1 - . 

1-75- 



.... 1 
1 - - 2 - 
1114 1 



1 - Very Poor 

2 - Poor 

3 - Fair/Average 

4 - Good 

5 - Very Good 



6 - Works Well in Groups 

7 - Compare Results with Academic 

Achievement Scores at Grade Level 

8 - Results Have Limited Use 

9 - Easy to Administer/Score 

10 - Hard to Administer/Score 

1 1 - Administration Time Consuming 

12 - Need Large Room to Administer 

13 - State Recommended* 

14 - Not Appropriate for Certain 

Disabilities 

15 - Meets State Requirements 

16 - Expensive 
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* There is no state recommendation for assessment instruments. 
Source: Kapes et. a ., 1 991 
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^ Rgures 10-3 through 10-6 give the results of a similar survey taken 
in Pennsylvania but in nnore detail. The tests are classified by area 
assessed and population for which they are considered appropriate. 

Rgure 10-3 (next page): Aptitude Assessment Instalments 

»3P Rgure 10-4 (page 30): Interest Assessmerrt Instruments 

« Rgure 10-5 (page 31): Learning Styles Assessment Instruments 

^ Rgure 10-6 (page 31): , Combination Assessment Instruments 

Source: Spedal Vocational Educational Sendees in Pennsylvania 
Director, Eleanor Bicanich 
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APTITUDE ASSESSMENT INSTRUMENTS RATING SHEET 



The numbers are averages tabulated from ratings collected from counselors on their 
recommendation (on a scale of 1 through 5 with 1 being first choice) for using that 
particular instrument with the population noted above each column. 



Trtltf 


Disad- 


LEP 
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Visually 


Hearing 


Phys. 


Mentally 


Disabled 






vantage 




Disabi. 


Impair. 


Impair. 
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Mild 


Mod 


Student 


Aptitude BaMd C«re«r Dect«ion Test (ABCD) 


1 
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1 


1 






1 
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1 
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Bennett Hand-Too4 Dextenty Te«t 
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4 


4 
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4 


Career Ability Placement Survey (CAPS) 


2 


2 
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2 


2 


2 
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Crawford Smail Parts Dexterity Test 




















Clerical Abilities Battery 




















Computer Aptitude, Literacy, and Interest Profile (CAUP) 
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3 


3 


3 


3 


3 
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Computer Operator Aptitude Battery (COAB) 




















Computer Programmer Aptitude Battery (CPAB) 




















DtfferentiaJ Aptitude Test (DAT) 




















Flanagan Industrial Test 




















Genera] Aptitude Test Battery (GATB) 
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McCamxi-Dlal Evaluation System (MDS) 
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Microcomp. Eval. and Screening and Assessment (MESA) 
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Minnesota Rate of Manipulation Test 




















Minnesota Spatial Relations Test 
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Short Employment Test (SET) 




















Short Test of Clerical Ability (STCA) 




















Skills Assessment Module (SAM) 
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Talent Assessment Program (TAP) 
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Wide Range Employabllity Sample (WREST) 





















Figure 10-3 



Sourco: Special Vocational Educational Services in PA, Director Eleanor Bicanich 
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IhTTEREST ASSESSMEhfT INSTRUMENTTS RATING SHEET 



The numbers are averages tabulated from ratings collected from counselors on their 
recommendation (on a scale of 1 through 5 with 1 being first choice) for using that 
particular instrument with the population noted above each co'umn. 
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LEARNING STYLES ASSESSMENT INSTRUMENTS RATING SHEET 



The numbers are averages tabulated from ratings collected from counselors on their 
recommendations (on a scale of 1 through 5 with 1 being first choice) for using that 
particular instrument with the population noted above each column. 
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Figure 10-5 

Source: Special Vocational Educational Sorvices in PA, Director Eleanor Bicanich 

COMBINATION ASSESSMENTT INSTRUMENTS RATING SHEET 

Instruments that include both aptitude and interest assessments, and sometimes learning 

styles. 

The numbers are averages tabulated from ratings collected from counselors on their 
recommendation (on a scale of 1 through 5 with 1 being first choice) for using that 
particular instrument with the population noted above each column. 
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Figure 10-6 

Source: Special Vocational Educational Services in PA, Director Eleanor Bicanich 
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The last survey of this representative overview was done by the Multi- 
cultural Task Force of the Vocational Evaluation and Work Adjustment 
Association (VEWAA). The Task Force was created and the survey taken 
to get the information needed for setting long-term goals that are sensitive 
to cultural diversity. 



The results were shared with us still in their rough draft form, but from the 
450-f responses already calculated, more than 83% put " Multi-Cultural 
Evaluation and Assessment' at the top of the list of Training Priorities. As 
to the question of what the professionals already working in the field felt 
was needed for improving vocational evaluation for multi-cultural individuals, 
a broad range of needs was indicated: 

* "Culturally-sensitive version of commercial systems including SAGE, 
Valpar, Mesa, and Apticom" 

* "Culture-fair GATB" 

* "Assess subsistence lifestyle (Alaskan Native fishing, carving, 
gathering, etc.)" 

* "Urban vs. rural norms" 

* "Brochures on values and customs" 

* "Culture-free mechanical aptitude tests" 

* 'Tests sensitive to socio-economic norms" 

* 'Tests developed by someone other than white European males" 
Source: Norma Colyer, Task Force Chairperson 
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WHERE? 




Entities That Offer Products On Assessing 
Occupational Competencies and Academic Skills 



American Association for Vocational Instructional Materials (AAVIM) 

745 Gaines School Road 

Athens, GA 30605 

(404) 543-7557 

1-800-228-4689 

George W. Smith, Interim Executive Director 

AAVIM, a consortium of states' department of education representatives, is 
a nonprofit educational association. AAVIM produces vocational training 
materials, including texts, manuals, workbooks, computer software, and 
videos. Also available for sale through AAVIM is a set of modules designed 
to assist vocational educators in the process of helping persons with 
limited-English proficiency in occupational programs. 



American College Testing (ACT) 
Work Keys 
P.O. Box 168 
Iowa City, lA 52243 
(319) 337-1000 

Work Keys, a new initiative underway at ACT, is a national system for 
competency-based assessment and teaching of employability skills. 
Learners entering the Work Keys system will be assessed to identify the 
performance level at which they demonstrate competencies for each of the 
Work Keys skills. 
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California Adult Student Assessment System (CASAS) 
2725 Congress Street 
Suite 1-M 

San Diego, CA 92110 
(619) 298-4681 

A consortium of education institutions in California that developed an 
assessment system that reflects the competency-based curricula used by 
local programs. The system will help (1) place adults in ESL and Adult 
Basic Education programs, as well as vocational and high school diploma 
programs, (2) measure student achievement, and (3) certify competency 
attainments. 



Center for Occupational Research and Development (CORD) 

601 C Lake Air Drive 

Waco, TX 76710 

(817) 772-8756 

(512) 323^779 (in Austin, TX) 

1-800-231-3015 

A variety of products and sen/ices are available from CORD, a nonprofit, 
public-service organization dedicated to the advancement of 
vocational/technical education. Spanning secondary, postsecondary, and 
adult training and education, CORD provides services, forecasts, strategies, 
curricula, and materials that are used to promote a more productive and 
competitive workforce. 



Educational Testing Service 
Rosedale Road 
Princeton, NJ 08541 
(609) 921-9000 

The Educational Testing Service (ETS) has a separate division, the Center 
for Occupational and Professional Assessment (COPA), that focuses on job 
related assessment programs and services. Most COPA tests are multiple 
choice, but the ETS also uses other assessment techniques such as essay 
tests, performance ^ssts, and simulations. 
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Jostens Learning Corporation 
6170 Comer^one Court East 
San Diego, CA 92121 
Contact: Marilyn Johnson 

Jostens Learning develops integrated learning systenns. These are 
networked systems of multiple computers delivering comprehensive 
educational courses for K-12 school students and adult learners. Many of 
Jostens' learning products use a guided exploration format utilizing 
animated learning activities which are designed for integration with 
classroom instruction. Portfolio assessment methods are being developed 
for many of Jostens Learning's programs to supplement other skill-based 
methods of assessment. 



National Occupational Competency Testing Institute (NOCTI) 

409 Bishop Hall 

Ferris State University 

901 South State 

Big Rapids, Ml 49307-2295 

(616) 796-4695 

1-800-334^83 

The National Occupational Competency Testing Institute provides 
occupational competency examinations and related services to both the 
United States and world communities, as well as NOCTI teacher (TOCT), 
student (SOVAT), and industrial (lOCT) testing products. 



Vocational-Technical Education Consortium of States (V-TECS) 
1866 Southem Lane 
Decatur, GA 30033-4097 
1-800-248-7701 

Since its founding in 1 973, the primary activity of the Vocational-Technical 
Education Consortium of States (V-TECS) has been to define and analyze 
occupations for the purpose of creating competency-based curriculum and 
programs. V-TECS has been developing competency-based assessment 
instruments based on performance objectives, and sponsors an annual 
conference, the National Conference on Competency-based Testing and 
Performance Standards for Vocational-Technical Education. During the last 
two years, development of the V-TECS Data System, a software package 
designed for storing and retrieving V-TECS material, has been one of their 
major efforts. 
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Surveys Used In This Chapter 



Kapes, Jerome et.al.. " Career Assessment Instruments Used With Students 
With Special Needs in Texas" (1991). Paper presented at the American 
Vocational Association convention, Los Angeles, CA., December, 1991, 

Special Vocational Education Services In Pennsylvania 

Director: Eleanor Bicanich 

Pennsylvania State University McKeesport Campus 

101 Ostermayer 

University Drive 

McKeesport, PA 15132 

Tel.: (412) 675-9065 

Fax: (412) 675-9067 

Vocational Evaluation and Work Adjustment Association (VEWAA) 
% National Rehabilitation Association 
633 W. Washington Street 
Alexandria, VA 22314 




Articles and Books Reflecting Research and Ideas 
About Standardized Tests and Their Usage: 



Cloud, Nancy, "Educational Assessment", In Hamayan & Damico (eds.), 
Limiting Bias in the Assessment of Bilingual Students . (1991). Austin, 
Texas: Pro-Ed. 

Collier, N., "Comparison of Acculturation and Education of Characteristics 
of Referred and Non-Referred Culturally and Linguistically Different 
Children", (March, 1987). Paper presented at Annual Meeting of the 
National Association for Bilingual Education, Denver, Colorado. 

Council of Chief State School Officers, Resource Center on Educational 
Equity, Summary of Recommendations and Policy Implications for 
Improving Assessment for LEP's , (1991). Washington, D.C.: Author. 
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SUMMARY 



I've often thought there ought to be a manual to hand to Me kids, 
telling them what kind of a planet they're on ... . And one thing I 
would really like to tell them about is cultural relativity. I didn't learn 
until I was in college about ail the other cultures, and I should have 
learned that In the first grade. A first grader should understand that 
his or her culture isnt a rational invention; that there are thousands of 
other cultures and they ail work pretty well; that ail cultures function 
on faith rather than truth; that there are lots of altematives to our own 
society. Cultural relativity is defensible and attractive. 

It's a source of hope. 

It means we don't have to continue this way if we don't like it 

Kurt Vonnegut, Jr. 



It is never too late to take positive steps to institute change! 



As an educator working with diverse students on a daily basis, you more 
than many others, know first hand the dynamics of constant transformation, 
reconfiguration, and the unending ebb and flow of your students* growth 
and needs. No two students are alike; no two students can be fully 
understood by using just one set strategy for assessment. 

Differences exist among all individuals regardless of ethnic background, 
country of origin, language spoken, gender, race, age, or family economic 
situation. 
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If we can begin with this premise, and accept individual differences as a 
beneficial characteristic and dynamic in an educational and work setting, 
then we can more easily move to the next step: 



Look for positive ways to change what we do so that we can better 
provide learning, guidance, and progression into the world of work. 



Experiencing achievement through gaining knowledge is one of the most 
exciting and rewarding activities individuals can discover. Learning and 
evaluating what has been learned is a very fluid task, often with no well- 
defined beginnings or endings; just a continuum of different rates and levels 
of benchmarks met, achievements accomplished. 



Yet, our instructional system, requirements for accountability at the local, 
state, and national levels, and workplace competencies, force us to 
continually try to pinpoint learning and achievement. 



Tests and other assessment measures are the way we attempt to 
accomplish this difficult task. Test results cannot, and should not always 
be ignored, of course, but test limitations must be consistently recognized. 



If you acknowledge the limitations of tests, rather than the perceived 
or real limitations of the individuals taking the tests, then you can 
modify not only the tests themselves, but the entire mind-set that 
accompanies assessment in general. 



This has been the goal and mission of this Handbook. The modifications and 
alternative procedures described herein represent a range of potential 
strategies and tools. We provide vocational educators with a selection of 
assessment techniques to obtain more APPROPRIATE, ACCURATE, and 
COMPREHENSIVE assessment results for limited English proficient individuals. 
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We purposefully made the effort to include assessment modifications that 
stretch from the general to specific. By offering a broad spectrum, we hope 
to increase the possibilities of impacting the greatest number of programs, 
and also show that no one strategy is the right answer for every student or 
situation. 



Some of the suggestions are merely taking what many teachers are already 
doing, and just doing it a little differently, in ways that better diversify 
procedures and results. Others might require taking a hard, honest look 
at needed changes beyond just the classroom, and revising curriculum 
and accountability structures that serve as barriers to certain students. 



This Handbook has been intended as a clearinghouse of experiences that 
will show how LEP students can actively contribute their productive part to 
an educational process of growth. Assessment, be it access into a 
program, on-going evaluation, or exit criteria, should be a major catalyst for 
opportunities, a key element in promoting development. !t is the constant 
challenge that reminds us what teaching is all about. 



Our goal is to enhance the quality and excitement, and value of 
learning experiences, and assessment should be assessed by its 
contribution to this goal. 

Dr. Sylvia Johnson. Howard University* 




* Testimony presented before the U.S. House of Representatives, Committee on Education and Labor, 
Subcommittee on Elementary, Secondary, and Vocational Education, February 18, 1993. 
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TEACHER'S NOTES 



Ideally, you have been making notations in the margins as you have been 
reading through this Handbook. 



To assist you in pulling together your thoughts, we end the Handbook with 
some suggested Reflections on what you have read. It is our hope that the 
ideas collected in this Handbook will serve as viable scenarios that go 
beyond words and become actions. Perhaps by summarizing here, you can 
form the beginning blueprint for constructing more accurate, appropriate, 
and comprehensive assessment for you and your students. Good Luck! 
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Things Noted That I Would Like To Do: 
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Sources To Follow Up On: 



People I Need To Talk With About Implementing Changes In My 
Instructional/Assessment Process: 
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